Adaptable integrated energy control system for electrosurgical tools in robotic surgical systems

ABSTRACT

A user interface for a surgical system can include a display configured to output video images of a remote surgical site at which one or more electrosurgical instruments of the surgical system are deployed; and a graphical user interface configured to be output on the display with the video images. The graphical user interface may comprise a visual indication of a state of the one or more electrosurgical instruments that indicates a state of the one or more electrosurgical instruments being ready for activation to deliver energy or actively delivering energy.

This application is a continuation application of U.S. application Ser.No. 15/161,158 (filed May 20, 2016), which is a continuation applicationof U.S. application Ser. No. 14/642,163 (filed Mar. 9, 2015, now U.S.Pat. No. 9,375,288), which is a continuation of U.S. application Ser.No. 14/497,712 (filed Sep. 26, 2014, now abandoned), which is acontinuation of U.S. application Ser. No. 13/800,856 (filed Mar. 13,2013, now U.S. Pat. No. 8,862,268), which is a divisional of U.S.application Ser. No. 12/400,653 (filed Mar. 9, 2009, now U.S. Pat. No.8,423,182), each of which is incorporated herein by reference in itsentirety.

FIELD

The embodiments of the invention are generally related to integratedcontrol of equipment that supports tools and the equipment to toolinterface in robotic surgical systems.

BACKGROUND

Robotic surgical systems may have multiple robotic arms to which aplurality of robotic surgical tools (also referred to as roboticsurgical instruments) may be coupled. One such category of roboticsurgical tools is electrosurgical tools which includes a monopolarelectrosurgical tool or a bipolar electrosurgical tool as well asharmonic, laser, ultrasound tools. Another category of robotic surgicaltools is tissue manipulation tools which may have articulated endeffectors (such as jaws, scissors, graspers, needle holders, microdissectors, staple appliers, tackers, suction/irrigation tools, clipappliers, or the like) or non-articulated end effectors (such as cuttingblades, irrigators, catheters, suction orifices, or the like) withoutelectrosurgical elements. While electrosurgical tools are mechanicallycoupled to a robotic arm to control its movement, they are also coupledto electrosurgical energy generating units (ESUs) so that energy may beapplied to tissue at or near its end effectors.

Electrosurgical tools in a robotic surgical system may be mechanicallycontrolled by one or both of a surgeon's left and/or right hands andelectrically controlled to deliver energy to tissue by a surgeon's foot.When viewing an image of an electrosurgical tool and tissue on a displaydevice captured through a camera, it may be difficult to see the effectof an inadvertent application of energy to the tissue. While a singlemis-application of energy to tissue may cause some damage, increasingdamage is likely to be caused to tissue with repetitive mis-applicationof energy.

The inadvertent application of energy to tissue may be caused by amis-positioned foot over an incorrect pedal. To avoid a mis-positionedfoot, a surgeon may have to look away from the display device to see hisfoot and properly position it over the proper one of a plurality of footpedal switches. The inadvertent application of energy to tissue may alsobe caused surgeon confusion as to which of a plurality ofelectrosurgical tools in the surgical site is being energized. Moreover,if a surgeon's concentration is narrowly focused on the wrong surgicaltool, he may not see the tissue damage being done by itsmis-application. Energizing the proper electrosurgical tool in asurgical site can increase surgical efficiency and avoid excessivetissue damage to a patient.

Furthermore with additional types of robotic surgical tools being usedwith controllable equipment, it has become more difficult for a surgeonto simultaneously control all of the desired instruments.

BRIEF SUMMARY

The embodiments of the invention are summarized by the claims thatfollow below.

BRIEF DESCRIPTIONS OF THE DRAWINGS

FIG. 1A is a block diagram of a first robotic surgery system to performminimally invasive robotic surgical procedures using a roboticelectrosurgical tool.

FIG. 1B is a block diagram of a second robotic surgery system to performminimally invasive robotic surgical procedures using a roboticelectrosurgical tool.

FIG. 2A is a perspective view of a robotic surgical manipulator with aplurality of robotic surgical arms at least one of which includes arobotic electrosurgical tool.

FIG. 2B illustrates mounting of the robotic electrosurgical tool to anadapter of the robotic surgical arm.

FIG. 2C illustrates a top view of the adapter of the robotic surgicalarm of FIG. 2C to which the robotic electrosurgical tool may be mounted.

FIG. 2D illustrates a back side of an exemplary robotic electrosurgicalinstrument or tool that interfaces to a robotic surgical arm.

FIG. 3A is a perspective view of a robotic surgical master controlconsole (surgeons console).

FIG. 3B is a perspective view of an exemplary gimbaled control inputwrist pivotally supporting a master grip control handle (also referredto as a master grip control input) for the robotic surgical mastercontrol console of FIG. 3A to control robotic surgical tools including arobotic electrosurgical tool.

FIG. 3C is a cross-sectional view schematically illustrating the mastergrip control handle (also referred to as a master grip control input)pivotally coupled to the control input wrist of FIG. 3B with elementscoupled to the feedback generator to provide haptic/tactile userfeedback.

FIG. 4 is a perspective view of the stereo viewer of the master controlconsole of FIG. 3A illustrating a graphical user interface overlaid ontovideo images of a surgical site.

FIGS. 5A-5C illustrate images displayed by a display device including agraphical user interface overlaid onto video images of a work site inaccordance with one embodiment of the invention.

FIGS. 6A-6B illustrate magnified images of master control iconsincluding a master grip map and a pedal map.

FIGS. 7A-7F are various views of an integrated pedal system of thesurgeon's control console.

FIG. 8 is a perspective view of the integrated pedal system assembled tothe surgeon's control console.

FIG. 9 is a perspective view of a workspace in the surgeon's consoleshowing a left master controller and a right master controller.

FIG. 10 is a magnified perspective view of the right master grip andright control input wrist of the right master controller shown in FIG.9.

FIG. 11A is a magnified perspective view of the ergonomic control panelin the surgeon's console shown in FIG. 9.

FIG. 11B is a top view of the liquid crystal display (LCD) touchpadscreen in the surgeon's console shown in FIG. 9.

FIG. 12 illustrates waveform diagrams of different signal profiles ofsignals that may be used to provide user feedback; the handedness—leftor right hand (or sidedness—left or right side) of the user feedback;and differential user feedback with different signal patterns.

FIGS. 13A-13B are flowchart diagrams illustrating methods for enhancedactivation of an electrosurgical device.

FIG. 14 is a flow chart illustrating a method for generating a graphicaluser interface (GUI) and displaying the GUI with video images to providevisual feedback to a user.

FIG. 15 is a flow chart of a method of swapping energy handedness fromone surgical tool in one side to another surgical tool in an oppositeside.

FIG. 16 illustrates a block diagram of an integrated robotic surgicalcontrol system.

FIGS. 17A-17B illustrate simplified block diagrams of alternateembodiments of the integrated robotic surgical control system.

FIGS. 18A-18B illustrate a block diagram of the pedal control logic andalternate input/output signal flow corresponding to the alternateembodiments of the integrated robotic surgical control system. shown inFIGS. 17A-17B.

FIG. 19 illustrates a perspective view of a smart cable.

FIGS. 20A-20C illustrate schematic diagrams of how a switch may beadapted to different types of control lines for different remotecontrolled equipment.

FIG. 21 illustrates a block diagram of a computer system including thefunctionality of the integrated user interface controller.

Note that these figures are for illustration purposes and do notnecessarily reflect the actual shape, size, or dimensions of objectsbeing illustrated.

DETAILED DESCRIPTION

In the following detailed description of the embodiments of theinvention, numerous specific details are set forth in order to provide athorough understanding of the embodiments of the invention. However, itwill be obvious to one skilled in the art that the embodiments of theinvention may be practiced without these specific details. In otherinstances well known methods, procedures, components, and circuits havenot been described in detail so as not to unnecessarily obscure aspectsof the embodiments of the invention.

Introduction

Robotic surgery may be used to perform a wide variety of surgicalprocedures, including but not limited to open surgery, neurosurgicalprocedures (such as stereotaxy), endoscopic procedures (such aslaparoscopy, arthroscopy, thoracoscopy), and the like. During theserobotic surgical procedures, surgeons may use high voltage, low currentelectrical energy of various wave forms to perform such tasks ascautery, cutting tissue, or sealing a vessel. Electrical energy supplydevices (referred to as electrosurgical generating units ESU) arecoupled to surgical instruments and typically activated by a foot pedalswitch of a foot pedal. One or more foot pedals in a surgeon's consoleand their corresponding switches may be used to activate theseelectrical energy supply devices. The foot pedal switches in thesurgeon's console replace the original equipment manufacturers (OEM)foot pedal switches that are packaged with the ESUs as standardequipment. An OEM pedal directly connects to a specific ESU and may notbe used to control other equipment.

In one embodiment of the invention, a method for a minimally invasivesurgical system is disclosed including reading first tool informationfrom a storage device in a first robotic surgical tool mounted to afirst robotic arm to at least determine a first tool type; readingequipment information about one or more remote controlled equipment forcontrol thereof; comparing the first tool information with the equipmentinformation to appropriately match a first remote controlled equipmentof the one or more remote controlled equipment to the first roboticsurgical tool; and mapping one or more user interface input devices of afirst control console to control the first remote controlled equipmentto support a function of the first robotic surgical tool.

In another embodiment of the invention, an adaptable integrated userinterface controller for a control console is disclosed. The adaptableintegrated user interface controller includes mapping logic to couple toone or more user interface input devices to receive control signals inresponse to selection of the one or more user interface input devices;and an adaptable input/output (I/O) interface coupled to the mappinglogic. The adaptable I/O interface couples to one or more pieces ofremote controlled equipment. The adaptable I/O interface reads equipmentinformation of the one or more pieces of remote controlled equipment andadapt signal levels of the one or more user interface input devices tosignal levels of the one or more pieces of remote controlled equipment.The mapping logic selectively maps the one or more user interface inputdevices to the one or more pieces of remote controlled equipment inresponse to the equipment information regarding the one or more remotecontrolled equipment, tool information of one or more robotic surgicaltools controllable by the control console, and an active tool controlsignal selected by a user. The active tool control signal swaps controlof at least one user interface device to the remote controlled equipmentbetween the one or more robotic surgical tools.

In yet another embodiment of the invention, another adaptable integrateduser interface controller for a control console is disclosed. Theadaptable integrated user interface controller in this case includesmapping logic to couple to one or more user interface input devices toreceive control signals in response to selection of the one or more userinterface input devices; and a tool interface coupled to one or morerobotic surgical tools. The tool interface reads tool information fromone or more robotic surgical tools that are controllable by the controlconsole and reads equipment information of one or more pieces of remotecontrolled equipment from the one or more intelligent robotic surgicaltools. The mapping logic selectively maps the one or more user interfaceinput devices to the one or more intelligent robotic surgical tools toremotely control the one or more remote controlled equipment through theintelligent one or more robotic surgical tools. The selective mappingperformed by the mapping logic is in response to the equipmentinformation of the one or more pieces of remote controlled equipment,tool information of the one or more intelligent robotic surgical tools,and an active tool control signal selected by a user. The active toolcontrol signal swaps control of at least one user interface device tothe remote controlled equipment between the one or more intelligentrobotic surgical tools.

In still another embodiment of the invention, a minimally invasivesurgical system is disclosed. The minimally invasive surgical systemincludes one or more robotic surgical tools, an integrated userinterface controller, and a processor coupled to the integrated userinterface controller and the one or more robotic surgical tools. Theintegrated user interface controller has a first interface coupled toone or more pieces of remote controlled equipment and a second interfacecoupled to one or more user interface input devices of a first controlconsole. The integrated user interface controller further has mappinglogic to map the one or more user interface input devices to control oneor more of the one or more pieces of first remote controlled equipment.In response to stored program instructions in a storage device coupledto the processor, the processor becomes configured to read first toolinformation from a storage device in a first robotic surgical toolmounted to a first robotic arm to at least determine a first tool type;read equipment information about one or more pieces of remote controlledequipment for control thereof; compare the first tool information withthe equipment information to appropriately match the one or more remotecontrolled equipment to the first robotic surgical tool; and control themapping logic to map the one or more user interface input devices of thefirst control console to control the first remote controlled equipmentto support a function of the first robotic surgical tool.

Robotic Surgical Systems

Robotic surgery generally involves the use of a robot manipulator thathas multiple robotic manipulator arms. One or more of the roboticmanipulator arms often support a robotic surgical tool or instrumentwhich may be an electrosurgical tool or a non-electrosurgical tool. Oneor more of the robotic manipulator arms are often used to support asurgical image capture device such as an endoscope (which may be any ofa variety of structures such as a laparoscope, an arthroscope, ahysteroscope, or the like), or, optionally, some other imaging modality(such as ultrasound, fluoroscopy, magnetic resonance imaging, or thelike). Typically, the robotic manipulator arms will support at least tworobotic surgical tools corresponding to the two hands of a surgeon andone image capture device.

Referring now to FIG. 1A, a block diagram of a robotic surgery system100A is illustrated to perform minimally invasive robotic surgicalprocedures using robotic electrosurgical tools 101A and 101B. Each ofthe robotic electrosurgical tools 101A and 101B are robotic endoscopicsurgical instrument that are manipulated by a slaved robotic manipulatorand remotely controlled by control signals received from a mastercontrol console. In contrast, manual endoscopic surgical instruments aredirectly controlled by hand. Robotic electrosurgical tool 101A is abipolar electrosurgical tool. Robotic electrosurgical tool 101B is amonopolar electrosurgical tool.

A user or operator O (generally a surgeon) performs a minimally invasivesurgical procedure on patient P by manipulating input devices at amaster control console 150. The master control console 150 may also bereferred to herein as a control console, a surgeon console, or a masterconsole. A computer 151 of the console 150 directs movement ofrobotically controlled endoscopic surgical instruments (generallynumbered 101), effecting movement of the instruments using a roboticsurgical manipulator 152. The robotic surgical manipulator 152 may alsobe referred to as robotic patient-side cart system or simply as a cart.The robotic surgical manipulator 152 has one or more robotic arms 153.Typically, the robotic surgical manipulator 152 includes at least threerobotic manipulator arms 153 supported by linkages, with a central armsupporting an endoscopic camera 101C and the robotic surgical arms 153to left and right of center supporting tissue manipulation tools and therobotic surgical tool 101A.

An assistant A may assist in pre-positioning of the robotic surgicalmanipulator 152 relative to patient P as well as swapping tools orinstruments 101 for alternative tool structures, and the like, whileviewing the internal surgical site via an assistant's display 154. Theimage of the internal surgical site shown to A by the assistant'sdisplay 154 and operator O by surgeon's console 150 is provided by oneof the surgical instruments 101 supported by the robotic surgicalmanipulator 152.

Generally, the robotic arms 153 of robotic surgical manipulator 152include a positioning portion and a driven portion. The positioningportion of the robotic surgical manipulator 152 remains in a fixedconfiguration during surgery while manipulating tissue. The drivenportion of the robotic surgical manipulator 152 is actively articulatedunder the direction of the operator O generating control signals at thesurgeon's console 150 during surgery. The actively driven portion of thearms 153 is herein referred to as an actuating portion 158. Thepositioning portion of the robotic arms 153 that are in a fixedconfiguration during surgery may be referred to as positioning linkageand/or “set-up joint” 156, 156′.

To support the functionality of the electrosurgical robotic tools101A-101B, the robotic surgical system 100 may further include one ormore electrosurgical generators 102A-102B. The one or moreelectrosurgical generators 102A-102B are remotely controlled by themaster console 150 over the control cables 109A-109B by a surgeonoperating the master console.

The electrosurgical generator 102A is a bipolar generator. A pair ofwires 106A-106B couple between the bipolar electrosurgical generator102A and a bipolar electrosurgical robotic tool 101A. The pair of wirespair of wires 106A-106B may transfer the energy of the bipolarelectrosurgical generator 102A to a respective pair of end effectors ofthe bipolar electrosurgical robotic tool 101A to cauterize or sealtissue.

The electrosurgical generator 102B is a monopolar generator. A wire 107couples between the monopolar electrosurgical generator 102B and amonopolar electrosurgical robotic tool 101B. A ground wire 108 couplesbetween the monopolar electrosurgical generator 102B and patient P. Thewire 107 may transfer the energy of the monopolar electrosurgicalgenerator 102B to an end effector of the monopolar electrosurgicalrobotic tool 101B to cauterize or seal tissue. A monopolarelectrosurgical generator and a bipolar electrosurgical generator may becombined together into one electrosurgical generator 102A′ that can beremotely controlled by two sets of controls from the control console150. That is, a first set of controls of the equipment 102A′ can be usedto control one function of the remote controlled equipment to supply(e.g., monopolar electrosurgical energy) a first robotic surgical toolwhile a second set of controls of the equipment can be used to controlanother function of the remote controlled equipment to supply (e.g.,bipolar electrosurgical energy) a second robotic surgical tool. Theremote controlled equipment may also be referred to as remotecontrollable equipment or remote controlled supply equipment. Therobotic surgical tools that couple to the remote controlled equipment toreceive a supply may also be referred to as supply controllable tools.

Much of the description herein is directed to remote controlledelectrosurgical generators and control of the supply of electrosurgicalenergy to robotic electrosurgical tools. However, the description hereinis more general in that it is equally applicable to other types ofremote controlled supply equipment (e.g., laser generator) and supplycontrollable tools (e.g., laser surgical tool). The remote controlledsupply equipment may be used to supply vacuum, gasses, liquids, energy(e.g., electrical, laser, ultrasound), mechanical torques, mechanicalpush/pull forces, data signals, control signals, etc. to supportfunctions of other types of robotic surgical tools (e.g., ultrasound,lasers, staplers). For example, a robotic surgical tool may combine thefunction of laser cutting and ultrasound together that is supported by aremote controlled laser generator and a remote controlled ultrasoundgenerator, both of which can be remotely controlled from the surgeonconsole. For further detail, see U.S. application Ser. No. 12/060,112,entitled ROBOTIC SURGICAL TOOLS FOR LASER MARKING AND LASER CUTTING,filed by Matthew Williams et al. on Mar. 31, 2008, which is incorporatedherein by reference.

Referring now to FIG. 1B, a block diagram of a robotic surgery system100B is illustrated. The robotic surgery system 100B is similar to therobotic surgery system 100A but with a control cart 150B beingintroduced between the surgeon's console 150A and the patient side cart152. The control cart 150B includes a computer 151B, and optionally, anexternal monitor 154. To further control or support the robotic surgicaltools, the control cart 150B includes one or more pieces of remotecontrollable equipment 102A′-102N′.

One piece of remote controllable equipment 102A′ mounted in the controlcart may be an electrosurgical generator that combines a monopolarelectrosurgical generator and a bipolar electrosurgical generatortogether to supply electrosurgical energy to two electrosurgical tools101A-101B. A pair of wires 106A-106B couple between the electrosurgicalgenerator 102A′ for a bipolar electrosurgical robotic tool 101A. Thepair of wires pair of wires 106A-106B may transfer the energy of thebipolar electrosurgical generator 102A′ to a respective pair of endeffectors of the bipolar electrosurgical robotic tool 101A to cauterizeor seal tissue. A wire 107 couples between the electrosurgical generator102A′ and a monopolar electrosurgical robotic tool 101B. A ground wire108 (not shown in FIG. 1A, see FIG. 1B) is used to couple between theelectrosurgical generator 102A′ and a patient P.

A control cable 110 couples between the computer 151B of the controlcart 150B and the surgeon's console to control the surgical system,including the remote controllable equipment and the robotic arms androbotic surgical tools. A control cable 111 is coupled the computer 151Band the patient side cart 152 for the surgeon's console to control therobotic arms and robotic surgical tools through the control cart.

Smart cables 112A-112N may be respectively coupled between the one ormore pieces of remote controllable equipment 102A′-102N′ and thecomputer 151B in the control cart 150B. With these connections, thesurgeon's console can control the remote controllable equipment with itsfoot pedals and master controllers. In this manner, the control of theremote controllable equipment 102A′-102N′ may be integrated into thesurgeon's console. Its foot pedals and master controllers becomeintegrated control mechanisms that a surgeon may use to control everyaspect of the surgical system to make robotic surgery more efficient.Advanced user interfaces may be used to provide improved control andfeedback of operating the remote controllable equipment with the roboticsurgical tools.

Patient Side Cart (Robotic Surgical Manipulator)

Referring now to FIG. 2A, a perspective view of the robotic surgicalmanipulator 152 is illustrated. The robotic surgical manipulator 152 mayalso be referred to as a patient side cart (PSC).

The robotic surgical manipulator 152 has one or more robotic surgicalarms 153. The robotic arm 153C includes an electrosurgical robotic tool101A coupled thereto. The robotic surgical manipulator 152 furtherincludes a base 202 from which the robotic surgical instruments 101 maybe supported. More specifically, the robotic surgical instruments 101are each supported by the positioning linkage 156 and the actuatingportion 158 of the arms 153. It should be noted that these linkagestructures are here illustrated with protective covers 206, 208extending over much of the robotic arms. It should be understood thatthese protective covers 206, 208 are optional, and may be limited insize or entirely eliminated in some embodiments to minimize the inertiathat is manipulated by the servomechanism, and to limit the overallweight of robotic surgical manipulator 152.

Each of the robotic surgical tools 101A-101C, releasably couple to amoveable carriage 237 near an end of each robotic surgical arm. Eachmoveable carriage 237, with the robotic surgical tool mounted thereto,can be driven to translate along a linear guide formation 260 in theactuating portion 158 of the robotic surgical arms 153 in the directionof arrow 257.

The robotic surgical manipulator 152 generally has dimensions suitablefor transporting between operating rooms. It typically can fit throughstandard operating room doors and onto standard hospital elevators. Therobotic surgical manipulator 152 may have a weight and a wheel (or othertransportation) system that allows the cart to be positioned adjacent anoperating table by a single attendant. The robotic surgical manipulator152 may be sufficiently stable during transport to avoid tipping, and toeasily withstand overturning moments that may be imposed at the ends ofthe robotic arms during use.

Each of the robotic manipulating arms 153 preferably includes a linkagethat constrains the movement of the surgical tool 101 mounted thereto.More specifically, linkage includes rigid links coupled together byrotational joints in a parallelogram arrangement so that the roboticsurgical tools rotate around a point in space. At the point in space,the robotic arm can pivot the robotic surgical tool about a pitch axisand a yaw axis. The pitch and yaw axes intersect at the point, which isaligned along a shaft of robotic surgical tool. The shaft is a rotatablehollow tube that may have a number of cables of a cable drive system tocontrol the movement of the end effectors 212.

The robotic arm provides further degrees of freedom of movement to therobotic surgical tool. Along an insertion axis, parallel to the centralaxis of the shaft of the robotic surgical tool, the robotic surgicaltool may slide into and out from a surgical site as indicated by arrow257. The robotic surgical tool can also rotate about the insertion axis.As the robotic surgical tool slides along or rotates about the insertionaxis, the center point is relatively fixed with respect to the basepatient side cart 152. That is, the entire robotic arm is generallymoved in order to maintain or re-position back to the center point.

The linkage of the robotic arm may be driven by a series of motorstherein in response to commands from a processor or computer. The motorsin the robotic arm are also used to rotate and/or pivot the roboticsurgical tool at the center point around the axes. If a robotic surgicaltool 101 further has end effectors to be articulated or actuated, stillother motors in the robotic arm may be used to control the endeffectors. Additionally, the motion provided by the motors may bemechanically transferred to a different location such as by usingpulleys, cables, gears, links, cams, cam followers, and the like orother known means of transfer, such as pneumatics, hydraulics, orelectronics.

The robotic surgical tools 101 are generally sterile structures, oftenbeing sterilizable and/or being provided in hermetically sealed packagesfor use. As the robotic surgical tools 101 will be removed and replacedrepeatedly during many procedures, a tool holder could potentially beexposed to contamination if the interface directly engages the toolholder. To avoid contamination to a tool holder and possible crosscontamination between patients, an adaptor for coupling to roboticsurgical tools 101 is provided in a robotic arm of the robotic surgicalmanipulator.

Referring now to FIGS. 2B-2D, the mounting of the robotic surgical tool101A to an adapter 228 of the robotic surgical arm is now brieflydescribed.

The robotic surgical arm 153 may include an adapter 228 to which theelectrosurgical robotic tool 101A or other surgical tool 101 may bemounted. FIG. 2D illustrates a front side of an exemplary adapter 228.The front side of the adaptor 128 is generally referred to as a toolside 230 and the opposite side is generally referred to as a holder side(not shown).

FIG. 2B illustrates a back side of an exemplary electrosurgical robotictool 400 as the surgical robotic tool 101A. The robotic surgical tool400 includes an exemplary mountable housing 401 including an interfacebase 412 that can be coupled to the adapter 228 to mount the tool 400 toa robotic arm of a surgical robotic manipulator. The interface base 412and the adapter 228 may be electrically and mechanically coupledtogether to actuate the robotic surgical tool 400. Rotatably coupled tothe interface base 412 are one or more rotatable receiving members 418,also referred to as input disks. Each of the one or more rotatablereceiving members 418 includes a pair of pins 422A and 422B generallyreferred to as pins 422. Pin 422A is located closer to the center ofeach rotatable receive member 418 than pin 422B. The one or morerotatable receiving members 418 can mechanically couple respectively toone or more rotatable drivers 234 of the adapter 228. The roboticsurgical tool 101A may further include release levers 416 to release itfrom the adapter 228 and the robotic arm.

The interface base 412 may further include one or more electricalcontacts or pins 424 to electrically couple to terminals of anelectrical connector 242 of the adapter 228. One or more terminals ofthe electrical connector 242 that can couple to the electrical contactsor pins 424 of the tool may be used to make electrocautery connections,such as between an integrated controller and the tool and/or between thetool and electrosurgical generating units. The interface base 412 mayfurther include a printed circuit board 425 and one or more integratedcircuits 426 coupled thereto and to the one or more pins 424. The one ormore integrated circuits 426 store tool information that may be used toidentify the type of robotic surgical tool coupled to the robotic arm,so that it may be properly controlled by the master control console 150.

Referring to FIGS. 2B and 2D, a robotic electrosurgical tool orinstrument 400 is illustrated. The robotic electrosurgical tool orinstrument 400 includes a mountable housing 401, an elongated shaft 404having a proximal end and a distal end; and end effectors (not shown)coupled near the distal end of the shaft 404. The mountable housing 401includes an interface or tool base 412 coupled to the proximal end ofthe shaft 404. The mountable housing 401 may further include one or moreelectrical connectors 474A-474B, a cover 472, and one or more releaselevers 416. At the distal end of the shaft 404, a mechanical wrist (notshown) may be used to move the end effectors.

The interface or tool base 412 of the tool 400 can couple to an adapter228 so that it is removeably connectable to the robotic surgical system.Other surgical tools with the same type of tool base may also couple tothe adapter and then the robotic arm. During surgery, the adapter 228 iscoupled to the moveable carriage 237. Thus, with the electrosurgicaltool 400 mounted to the adapter 228, it can translate with the carriage237 along an insertion axis of the robotic surgical arm 153 as indicatedby arrow 257 in FIG. 2A. The tool base 412 includes receiving elementsor input disks 418 that releaseably couple through an adapter to arotatable driving element 234 that is mounted on the carriage 237 of therobotic arm assembly 153. The rotatable driving elements 234 of thecarriage 237 are generally coupled to actuators (not shown), such aselectric motors or the like, to cause selective angular displacement ofeach in the carriage 237.

When mounted to a robotic surgical arm 153, end effectors may have aplurality of degrees of freedom of movement relative to arm 153, inaddition to actuation movement of the end effectors. The end effectorsof the robotic surgical tool are used in performing a surgical operationsuch as cutting, shearing, grasping, gripping, clamping, engaging, orcontacting tissue adjacent a surgical site. With an electrosurgicaltool, a conductor electrically communicates with at least one of the endeffectors to deliver electrical energy to tissue clamped by the grippingjaws.

As shown in FIG. 2D, the tool base 412 may be enclosed by a cover 472 towhich one or more electrical connectors 474A-474B may be mounted. Theone or more electrical connectors 474A-474B can receive one or morecables 106A-106B, 107 to couple to an electrosurgical generator unit,such as the bipolar generator 102A, the monopolar generator 102B, or acombined monopolar/bipolar generator 102A′ illustrated in FIG. 1A. Oneor more wires within the tools electrically couple between theelectrical connectors 474A-474B and the electrodes at the one or moreend effectors of the tool. Alternatively, one or more terminals of theelectrical connector 242 that can couple to the electrical contacts orpins 424 of the tool may be used to make the electrocautery connectionsbetween the tool and the electrosurgical generating units.

The adapter 228 includes one or more rotatable drivers 234 rotatablycoupled to a floating plate 236. The rotatable drivers 234 areresiliently mounted to the floating plate 236 by resilient radialmembers which extend into a circumferential indentation about therotatable drivers. The rotatable drivers 234 can move axially relativeto floating plate 236 by deflection of these resilient structures.

The floating plate 236 has a limited range of movement relative to thesurrounding adaptor structure normal to the major surfaces of theadaptor. Axial movement of the floating plate helps decouple therotatable drivers 234 from a robotic surgical tool 101 when its releaselevers 416 are actuated.

The one or more rotatable drivers 234 of the adapter 228 maymechanically couple to a part of the surgical tools 101. Each of therotatable drivers 234 may include one or more openings 240 to receiveprotrusions or pins 422 of rotatable receiving members 418 of therobotic surgical tools 101. The openings 240 in the rotatable drivers234 are configured to accurately align with the rotatable receivingelements 418 of the surgical tools 101.

The inner pins 422A and the outer pins 422B of the rotatable receivingelements 418 respectively align with the opening 240A and the opening240B in each rotatable driver. The pins 422A and openings 240A are atdiffering distances from the axis of rotation than the pins 422B andopenings 240B so as to ensure that rotatable drivers 234 and therotatable receiving elements 418 are not aligned 180 degrees out ofphase from their intended position. Additionally, each of the openings240 in the rotatable drivers may be slightly radially elongated so as tofittingly receive the pins in the circumferential orientation. Thisallows the pins 422 to slide radially within the openings 240 andaccommodate some axial misalignment between the tool and the adapter228, while minimizing any angular misalignment and backlash between therotatable drivers 234 and the rotatable receiving elements 418.Additionally, the interaction between pins 422 and openings 240 helpsrestrain the robotic surgical tool 101 in the engaged position with theadapter 228 until the release levers 416 along the sides of the housing401 push on the floating plate 236 axially from the interface so as torelease the tool 101.

When disposed in a first axial position (away from the tool side 230)the rotatable drivers are free to rotate without angular limitation. Theone or more rotatable drivers 234 may rotate clockwise orcounter-clockwise to further actuate the systems and tools of therobotic surgical instruments 101. However, as the rotatable drivers moveaxially toward the tool side 230, tabs (extending radially from therotatable drivers) may laterally engage detents on the floating platesso as to limit the angular rotation of the rotatable drivers about theiraxes. This limited rotation can be used to help engage the rotatabledrivers the rotating members of the tool as the pins 422 may push therotatable bodies into the limited rotation position until the pins arealigned with (and slide into) the openings 240 in the rotatable drivers.

While rotatable drivers 234 are described here, other types of driversor actuators may be provided in the adapter 228 to actuate systems ortools of the robotic surgical instruments 101. The adapter 228 furtherincludes terminals of an electrical connector 242 to couple toelectrical contacts or pins 424 of surgical instruments 101 to make anelectrical connection as well.

The mounting of robotic surgical tool 101A to the adapter 228 generallyincludes inserting the tip or distal end of the shaft or hollow tube ofthe robotic surgical tool through a cannula (not shown) and sliding theinterface base 412 into engagement with the adapter 228, as illustratedin FIG. 2C. A lip 232 on the tool side 230 of the adaptor 228 slideablyreceives the laterally extending portions of the interface base 412 ofthe robotic surgical tool. A catch 244 of adapter 228 may latch onto theback end of the interface base 412 to hold the tool 101A in position.The protrusions or pins 422 extending from the one or more rotatablemembers 418 of the robotic surgical tool couple into the holes 240A-240B(generally referred to as holes or openings 240) in the rotatabledrivers 234 of the adapter 228.

The range of motion of the rotatable receiving elements 418 in therobotic surgical tool may be limited. To complete the mechanicalcoupling between the rotatable drivers of the adapter and the rotatablereceiving elements 418, the operator O at the surgical master controlconsole 150 may turn the rotatable drivers in one direction from center,turn the rotatable drivers in a second direction opposite the first, andthen return the rotatable drivers to center. Further, to ensure that thepins 422 enter openings 240 of rotatable drivers adapter 228, theadapter 228 and tool 101A mounted thereto may be moved together. Theadapter 228 and tool 101A mounted thereto may be moved to an initialposition so that the tip or distal end of the shaft or hollow tube isdisposed within a cannula (not shown).

To dismount and remove the robotic surgical tool 101A, the releaselevers 416 may be squeezed pushing out on the mountable housing 401 torelease the pins 422 from the holes 240 and the catch 244 from the backend of the interface base. The mountable housing 401 is then pulled upto slide the interface base 412 up and out from the adapter 228. Themountable housing 401 is continually pulled up to remove the tip ordistal end of the shaft or hollow tube out from the cannula 219. Afterthe robotic surgical tool 101A is dismounted, another robotic surgicaltool may be mounted in its place, including a new or freshly sterilizedelectrosurgical robotic tool 400.

As previously discussed, the robotic surgical tool 101A may include oneor more integrated circuits 426 to identify the type of robotic surgicaltool coupled to the robotic arm, such that it may be properly controlledby the master control console 150. However, the robotic surgical systemmay determine whether or not the robotic surgical tool is compatible ornot, prior to its use.

The system verifies that the tool is of the type which may be used withthe robotic surgical system 100. The one or more integrated circuits 426may signal to the computer 151 in the master control console 150 dataregarding compatibility and tool-type to determine compatibility as wellas control information. One of the integrated circuits 426 may include anon-volatile memory to store and read out data regarding systemcompatibility, the tool-type and the control information. In anexemplary embodiment, the data read from the memory includes a characterstring indicating tool compatibility with the robotic surgical system100. Additionally, the data from the tool memory will often include atool-type to signal to the master control console how it is to becontrolled. In some cases, the data will also include tool calibrationinformation. The data may be provided in response to a request signalfrom the computer 151.

Tool-type data will generally indicate what kind of tool has beenattached in a tool change operation. The tool-type data may includeinformation on wrist axis geometries, tool strengths, grip force, therange of motion of each joint, singularities in the joint motion space,the maximum force to be applied via the rotatable receiving elements,the tool transmission system characteristics including informationregarding the coupling of rotatable receiving elements to actuation orarticulation of a system within the robotic surgical instrument.

For example, the tool-type data might indicate that an electrosurgicalrobotic instrument 101A has been mounted to the robotic arm or not.Relevant to energy activation of an electrosurgical instrument,additional tool type data related to primary and/or secondary energysub-features may further be stored. For example, energy sub-features mayinclude what type of electrosurgical energy the tool may receive (e.g.,bipolar or monopolar cutting & monopolar coagulating), maximum peakenergy, minimum harmonic energy frequency, maximum harmonic energyfrequency, and whether or not a laser is also provided for cutting. Asnew energy or other types of modalities are introduced for roboticsurgical tools, its tool-type data can be readily stored andcommunicated to the robotic surgical system so that the system canadaptively control remote controllable equipment and multiple types ofrobotic surgical tools mounted to robotic arms of the robotic surgicalsystem.

Instead of storing all of the tool-type data in the one or moreintegrated circuits 426, most of the tool-type data may optionally bestored in memory or a hard drive of the computer 151 in the roboticsurgical system 100. An identifier may be stored in the one or moreintegrated circuits 426 to signal the computer 151 to read the relevantportions of data in a look up table store in the memory or the harddrive of the computer. The tool-type data in the look-up table may beloaded into a memory of computer 151 by the manufacturer of the roboticsurgical system 100. The look-up table may be stored in a flash memory,EEPROM, or other type of non-volatile memory. As a new tool-type isprovided, the manufacturer can revise the look-up table to accommodatethe new tool-specific information. It should be recognized that the useof tools which are not compatible with the robotic surgery system, forexample, which do not have the appropriate tool-type data in aninformation table, could result in inadequate robotic control over therobotic surgical tool by the computer 151 and the operator O.

In addition to the tool-type data, tool specific information may bestored in the integrated circuit 426, such as for reconfiguring theprogramming of computer 151 to control the tool. There may becalibration information, such an offset, to correct a misalignment inthe robotic surgical tool. The calibration information may be factoredinto the overall control of the robotic surgical tool. The storing ofsuch calibration information can be used to overcome minor mechanicalinconsistencies between tools of a single type. For example, thetool-type data including the tool-specific data may be used to generateappropriate coordinate transformations and servo drive signals tomanipulate the robotic arm and rotate the rotatable drivers 234. In thiscase, the integrated circuit 426 includes the information to set up thecontrol system to drive the end effectors in the tool to have a maximumjoint torque setting so that the jaws of a robotic gripping tool or arobotic electrosurgical tool can clamp to tissue with a maximum force.

Additionally, some robotic surgical tools have a limited life span. Toollife and cumulative tool use information may also be stored on the toolmemory and used by the computer to determine if the tool is still safefor use. Total tool life may be measured by clock time, by procedure, bythe number of times the tool has been loaded onto a holder, and in otherways specific to the type of tool. Tool life data is preferably storedin the memory of the tool using an irreversible writing process.

Master Control Console (Surgeon's Console)

Referring now to FIG. 3A, a perspective view of the robotic surgicalmaster control console 150, 150A is illustrated. The master controlconsole 150, 150A may also be referred to as the surgeon's console. Themaster control console 150 generates the control signals to controlsurgical tools (e.g., the electrosurgical robotic instruments) in asurgical site and medical equipment that supports the surgical tools(e.g., electrosurgical generators).

The master control console 150 of the robotic surgical system 100includes a binocular or stereo viewer 312, an arm-rest 314, a microphone315, a pair of master controllers 925L, 925R (generally referenced by325) for end effector input control, wrist input control, and arm inputcontrol within a workspace 316, one or more speakers 315L, 315R, andfoot pedals 318 (including foot pedals 318A-318B), and a viewing sensor320.

In one embodiment of the invention, the master control console 150further includes an integrated computer 151. In another embodiment ofthe invention, as shown in FIG. 1B, the computer may be distributed fromthe master control console as a computer 151B that is an element of thecentral control cart 150B. In either case, the computer 151, 151B mayinclude one or microprocessors 302 to execute instructions and a storagedevice 304 to store software with executable instructions that may beused to generate control signals to control the robotic surgical system100. The computer 151, 151B may further include a sound generator 317coupled to the microprocessors 302 and one or more speakers (leftspeaker 315L, right speaker 315R) to generate audible sounds; ahaptic/tactile feedback generator 321 coupled to the microprocessors 302and one or more vibrating devices at the master controls, the arm rest,and/or the foot pedals to generate vibrations; a graphicscontroller/generator 322 coupled to the microprocessors 302 and thestereo viewer 312 to generate a graphical user interface (GUI) and fusethe GUI data together with frames of camera image data of a surgicalsite for display on the display devices of the stereo viewer 312.

The viewer 312 has at least one display where images of a surgical sitemay be viewed to perform minimally invasive surgery. In a preferredembodiment of the invention, the viewer 312 is a stereo viewer with leftand right display devices 452L, 452R as illustrated in FIG. 4. FIG. 4illustrates a perspective view of the stereo viewer 312 of the mastercontrol console 150, 150A. To provide a three-dimensional perspective,the viewer 312 includes stereo images for each eye including a leftimage 450L and a right image 450R of the surgical site including anyrobotic surgical tools respectively in a left viewfinder 451L and aright viewfinder 451R. The images 450L and 450R in the viewfinders areprovided by the left display device 452L and a right display device452R, respectively. The display devices 452L, 452R may optionally bepairs of cathode ray tube (CRT) monitors, liquid crystal displays(LCDs), or other type of image display devices (e.g., plasma, digitallight projection, etc.). In the preferred embodiment of the invention,the images are provided in color by a pair of color display devices452L, 452R; such as color CRTs or color LCDs. A graphical user interfacemay be displayed like borders 461L, 461R around or near edges of each ofthe display devices 452L, 452R.

Referring now to FIGS. 3A-3C and FIG. 9, a left master controller 905Land a right master controller 905R (including master grip 325) in theworkspace 316 can be used to generate control signals for the patientside cart 152 to control the robotic arms and the surgical tools mountedthereto. The left master controller 905L and the right master controller905R are positioned in the workspace 316 disposed beyond the arm support314 and below the viewer 312.

When using the master control console 150, 150A, the operator O (surgeonor user) typically sits in a chair, moves his or her head into alignmentwith the viewer 312, and couples his/her fingers to the master grips 325of left master controller 905L and the right master controller 905R, onein each hand, while resting their forearms against the arm rest 314.This allows the master grips 325 of the left master controller 905L andthe right master controller 905R to be moved easily in the control space316 in both position and orientation to generate control signals.

Additionally, the operator O can use his feet to control the foot-pedalsto change the configuration of the surgical system and generateadditional control signals to control the robotic surgical instrumentsor other medical equipment coupled to the system.

To ensure that the operator is viewing the surgical site whencontrolling the robotic surgical tools 101, the master control console150 may include the viewing sensor 320 disposed adjacent the binoculardisplay 312. When the system operator aligns his or her eyes with thebinocular eye pieces of the display 312 to view a stereoscopic image ofthe surgical worksite, the operator's head sets off the viewing sensor320 to enable the control of the robotic surgical tools 101. When theoperator's head is removed the area of the display 312, the viewingsensor 320 can disable or stop generating new control signals inresponse to movements of the master grips in order to hold the state ofthe robotic surgical tools.

The computer 151 with its microprocessors 302 interprets movements andactuation of the left master controller 905L and the right mastercontroller 905R (and other inputs from the operator O or otherpersonnel) to generate control signals to control the robotic surgicalinstruments 101 in the surgical worksite. In one embodiment of theinvention, the computer 151 and the viewer 312 map the surgical worksiteinto the controller workspace 316 so it feels and appears to theoperator that the left master controller 905L and the right mastercontroller 905R are working over a surgical worksite.

Master Controllers with Control Input Grips

Referring momentarily now to FIG. 9, a perspective view of thecontroller workspace 316 at the surgeon's console 150, 150A isillustrated. The surgeon's console 150, 150A has a left mastercontroller 905L and a right master controller 905R. The left mastercontroller 905L includes a left control input arm 935L, a left controlinput wrist 952L and a left control input grip 925L. The right mastercontroller 905R includes a right control input arm 935R, a right controlinput wrist 952R and a right control input grip 925R.

Referring now to FIG. 3B, a perspective view of a control input wrist352 representative of the left control input wrist 952L and the rightcontrol input wrist 952R is illustrated. The master controllers 905L,905R at the surgeon console include a control input grip or master grip325 and a control input wrist 352 coupled together to a control arm (seecontrol input arms 935L, 935R in FIG. 9). The control input wrist 352 isa gimbaled device that pivotally supports the master grip 325 of themaster control console 150 to generate control signals that are used tocontrol the robotic surgical manipulator 152 and the robotic surgicaltools 101, including electrosurgical robotic tools 101A, 101B. A pair ofcontrol input wrists 352 for the left and right master controllers aresupported by a pair of control input arms (not shown in FIG. 3B) in theworkspace 316 of the master control console 150, 150A. The control inputwrist 352 includes first, second, and third gimbal members 362, 364, and366. The third gimbal member 366 is rotationally coupled to a controlinput arm (not shown) of the master control console 150, 150A.

The master grip 325 includes a tubular support structure 351, a firstgrip 350A, and a second grip 350B. The first grip and the second gripare supported at one end by the structure 351. The master grip 325 canbe rotated about axis G illustrated in FIGS. 3B-3C. The grips 350A, 350Bcan be squeezed or pinched together about the tubular structure 351. The“pinching” or grasping degree of freedom in the grips is indicated byarrows Ha, Hb in FIG. 3B and arrows H in FIG. 3C.

The master grip 325 is rotatably supported by the first gimbal member362 by means of a rotational joint 356 g. The first gimbal member 362 isin turn, rotatably supported by the second gimbal member 364 by means ofthe rotational joint 356 f. Similarly, the second gimbal member 364 isrotatably supported by the third gimbal member 366 using a rotationaljoint 356 d. In this manner, the control wrist allows the master grip325 to be moved and oriented in the workspace 316 using three degrees offreedom.

The movements in the gimbals of the control wrist 352 to reorient themaster grip in space can be translated into control signals to controlthe robotic surgical manipulator 152 and the robotic surgical tools 101.

The movements in the grips 350A, 350B of the master grip 325 can also betranslated into control signals to control the robotic surgicalmanipulator 152 and the robotic surgical tools 101. In particular, thesqueezing motion of the master grips 350A, 350B over their freedom ofmovement indicated by arrows Ha, Hb or H, may be used to control the endeffectors of the robotic surgical tools.

To sense the movements in the master grip 325 and generate controlssignals, sensors can be mounted in the handle 325 as well as the gimbalmember 362 of the control input wrist 352. Exemplary sensors may be aHall effect transducer, a potentiometer, an encoder, or the like.

Referring now to FIG. 3C, a cross-sectional view of the master grip 325and gimbal member 362 of the control input wrist 352 is illustrated.FIG. 3C provides an example as to how the master grip 325 can be mountedto the control input wrist 352 to sense the gripping and rotation of thehandle to control robotic surgical tools 101.

As illustrated in FIG. 3C, the exemplary gimbal member 362 includesbeveled gears 368 a, 368 b which can couple the rotational motion of themaster grip 325 to a roll sensor 370. The roll sensor 370 may use apotentiometer or encoder 370 b included in a roll motor 370 a to sensethe rotation. Alternatively, a separate roll sensor, such as apotentiometer, may be directly coupled to the shaft 380 to sense therotation of the master grip. In any case, a roll sensor senses the rollmotion of the master grip 325 and generates control signals in responsethereto to control the robotic surgical tools 101.

To sense a squeezing motion in the grips 350A, 350B of the master grip325, a remote sensing assembly 386 may be included by the gimbal member362. The first and second grips 350A, 350B are adapted to be squeezedtogether by a hand of an operator O so as to define a variable gripseparation. The grip separation may be determined as a function of avariable grip angle with an axis or as a function of a variable gripseparation distance, or the like. Alternative handle actuations, such asmovement of a thumbwheel or knob may also be provided in the handle tocontrol the robotic surgical instruments 101.

In the exemplary embodiment, the remote sensor assembly 386 includes acircuit board 394 on which a first and a second Hall effect sensors,HE1, HE2 are mounted. A magnet 396 is disposed distally beyond thecircuit board 394 and the Hall effect sensors. A magnetic mass 398 isaxially coupled to the proximally oriented surface 390 of a push rod384. Thus, the magnetic mass 398 moves (as shown by Arrow J) with thepush rod 384 and varies the magnetic field at the Hall effect sensors inresponse actuation of the grips 350A, 350B.

To translate the squeezing action of the grips 350A, 350B to the sensor386, the gimbal member 362 includes a push rod 384 within the tubularhandle structure 351. Each of the grips 350A, 350B pivot about arespective pivot 334 a, 334 b in the tubular handle structure 351.Urging links 335 a, 335 b respectively couple between the grips 350A,350B and a first end of the push rod 384. The squeezing action of thegrips 350A, 350B is translated into a linear motion on the push rod 384by means of urging links 335 a, 335 b as shown by arrow A in FIG. 3C. Asecond end of the push rod 384 couples to the sensor 386. As discussedpreviously, the magnetic mass 398 is axially coupled to the surface 390of the push rod 384 in order to sense the linear motion in the push rodand the squeezing motion of the grips 350A, 350B.

A biasing mechanism such as spring 392 applies a force against thesqueezing motion of the grips to return them to full open when the gripsare released. The biasing spring 392 may be a linear or non-linearelastic device biasing against the depression of grips 350A, 350B, e.g.,a single or multiple element assembly including springs or other elasticmembers. For example, spring 392 may comprise a concentric dual springassembly whereby one spring provides a “softer” bias response as thegrips 350A, 350B are initially depressed, and a second spring provides asuperimposed “firm” bias response as the grips 350A, 350B approach afully depressed state. Such a non-linear bias may provide a pseudoforce-feedback to the operator.

The master grip may be an ideal place to provide haptic/tactile feedbackto a user's hands through a vibrating device or mechanism. The magnet396 of the master grip may be an electro-magnet under control of thehaptic/tactile feedback generator as a vibrating device to generatevibrations in the master grip for haptic/tactile user feedback.Alternatively, another type of vibrating device may be an element of themaster grip and controlled by the haptic/tactile feedback generator. Thecontrol input wrist 325 may have such a vibrating device or mechanism399 coupled to the haptic/tactile feedback generator 321 to generatevibrations in the control input wrist that may be felt by a user. Avibrating device 355 coupled to the feedback generator 321 may be a partof the master grip near where the user's fingers make contact.Alternatively, pre-existing electric motors in each control input wristof the master controllers may be used to generate vibrations. A feedbacksignal may cause the motor to alternatively rotate back and forth over asmall arc to generate a vibration. For example, roll motor 370 a may becoupled to the haptic/tactile feedback generator 321 and rotated backand forth over one or a couple of segments of the encoder 370 b. U.S.Pat. No. 6,522,906 issued to Salisbury, Jr., et al. on Feb. 18, 2003;and U.S. Pat. No. 6,493,608 issued to Niemeyer on Dec. 10, 2002 alsodescribe using haptic feedback at the master controllers.

Referring now back to FIG. 9, an arm-support or arm-rest 314 in thesurgeon's console 150, 150A may include a left vibrating feedbackmechanism 936L and a right vibrating feedback mechanism 736R positionedin the arm rest to be under an area where a surgeon (user or operator O)may rest his left and right forearms. The left vibrating feedbackmechanism 936L and the right vibrating feedback mechanism 736R couple tothe haptic/tactile feedback generator 321 to receive a left feedbacksignal and a right feedback signal. In response to the feedback signalfrom the haptic/tactile feedback generator 321, either the leftvibrating feedback mechanism 936L or the right vibrating feedbackmechanism 736R can generate vibrations in the arm-rest 314 that may befelt by a user's forearm resting on the arm-rest.

It should be noted that a wide variety of vibration devices ormechanisms may be used to generate vibrations that the operator O (user,surgeon) may feel when operating the controls at the surgeons' console150, 150A.

Pedal System for Integrated Control

Referring now to FIG. 7A, a surgical console pedal system 700 isillustrated in a top view. The pedal system 700 includes a movable pedaltray 702, a lower left pedal assembly 704L, a lower right pedal assembly704R, a left vertical switch pedal assembly 706L, a right verticalswitch pedal assembly 706R, an upper level pedal assembly 708L, and anupper right level pedal assembly 708R. The pedal system 700 may includea drive assembly to move the moveable pedal tray 702 over a floor. Thepedal system 700 may further include a lift assembly to raise themoveable pedal tray above the floor when transporting the surgeonconsole and lower the pedal tray when its pedals are ready to be used bya user.

Each of the pedal assemblies may be assigned by the robotic surgicalsystem to control medical equipment that may couple to one or moresurgical tools. The functionality controlled by each pedal may becontext sensitive and switch depending upon the type of surgical toolsbeing controlled. In one embodiment, the pedal system is assigned tocontrol electrosurgical tools in response to one or more electrosurgicaltools being mounted to one or more of the robotic arms respectively. Therobotic surgical system may sense that one or more electrosurgical toolsare mounted to one or more of the robotic arms.

The movable pedal tray 702 has a base portion 711 and an upper tier orterrace portion 712 coupled to and elevated above the base portion 711.The pedal tray 702 has a center chair cut-out or opening 710 in a frontedge of the base portion 711 so that a wheel of a chair may roll on thefloor therein to appropriately adjust the position of the user at thesurgeon's console.

The lower left pedal assembly 704L and the lower right pedal assembly704R are positioned within the base portion 711 of the pedal tray. Theupper left pedal assembly 708L and the upper right pedal assembly 708Rare positioned within the upper tier or terrace portion 712 slightlyelevated above the lower left pedal assembly 704L and the lower rightpedal assembly 704R. The upper tier or terrace portion 712 may be U-likeshaped to receive the left vertical switch pedal assembly 706L in a leftside and the right vertical switch pedal assembly 706R in a right side.

The left vertical switch pedal assembly 706L is situated near the lowerleft pedal assembly 704L so that a user's left foot may movehorizontally to toggle the switch. The right vertical switch pedalassembly 706R is located near the lower right pedal assembly 704R sothat a user's right foot may toggle the switch by horizontal movement.The pedals 704L-704R and 708L-708R are horizontal switch pedals that maybe activated when a user's foot moves vertically downward.

Referring momentarily to FIG. 8, the surgeon's console pedal system 700is shown moveably coupled to the surgeon's console 150, 150A. Thesurgeon's console pedal system 700 is movable within an opening 804 ofthe base 801 of the surgeon's console 150, 150A as shown by the arrow802. The pedal tray 702 moves inward and outward over the floor withinthe opening 804 of the surgeon's console between left and right pontoons805L, 805R of the base 801 of the surgeon's console. This is so thepedal positions can be adjusted to custom fit a size of the surgeonseated at a chair near the surgeon's console. Digital numbersrepresenting the position of the pedal tray 702 with respect to theconsole 150, 150A may be stored in a storage device for later recall bythe surgeon. The left and right pontoons 805L, 805R include left andright brake pedals 850L, 850R, each a latchable pedal, that may be usedto lock the casters (not shown) of the surgeon console to keep itstationary.

Referring now to FIGS. 7B and 7F, a top view and a side view of thepedal system 700 are respectively illustrated. The pedal system 700includes a pair of left rollers 715L rotatably coupled to a left sideand a pair of right rollers 715R rotatably coupled to right side of themoveable pedal tray. In one embodiment of the invention, the pair ofleft rollers 715L and the pair of right rollers 715R respectively rollalong a left lift/guide rail 716L and a right lift/guide rail 716R toguide the pedal tray within the opening 804 of the base 801 when raisedabove the floor for transporting or moving the surgeon's console. Inanother embodiment of the invention, the pair of left rollers 715L andthe pair of right rollers 715R respectively roll along the left guiderail 716L and the right guide rail 716R to guide the pedal tray withinthe opening 804 of the base 801 when the pedal tray is near orconfigured to roll along the floor. Otherwise when the pedal tray islowered to the floor rolling upon rollers (e.g., see rollers 735R inFIG. 7F), the left guide rail 716L and the right guide rail 716R aresufficiently low so that the rollers the pair of left rollers 715L andthe pair of right rollers 715R need not roll along the left guide rail716L and the right guide rail 716R. At the bottom, the pedal trayincludes a pair of rollers 735R pivotally coupled to the right side ofthe pedal tray and a similar pair of rollers pivotally coupled to theleft side of the pedal tray to roll along the floor and move the pedaltray when lowered with the castor brakes of the surgeon console are set.

As mentioned previously, the pedal tray 702 may be vertically raisedabove the floor and lowered to the floor as indicated by the arrowhead739. The lift mechanism or assembly of the pedal system 700 includes alinkage on each side pivotally coupled between the lift/guide rails716L, 716R, each other and a frame 750 of a drive assembly of the pedalsystem. The lift assembly of the pedal system further includes a pair ofsprings 734 each with an end coupled to the linkage in each side toapply a force and raise the pedal tray above the floor. Each spring 734has an opposite end coupled to the surgeon's console schematicallyindicated as a ground link in FIG. 7F. In one embodiment, each spring734 may be coupled to the base 801 of the surgeon's console to push upon the pedal tray to raise it above the floor. In another embodiment,each spring may be coupled to the base 801 to pull up on the pedal trayto raise it above the floor. In one embodiment of the invention thespring 734 is a gas spring.

The springs 734 pre-load the linkage to raise the pedal tray above thefloor within the opening. To lower the pedal tray 702, a user pressesdown on either brake pedal 850L, 850R shown in FIG. 8 to press down onthe linkage and overcome the force applied by the springs 734. Eitherbrake pedal can be latched in place to hold the pedal tray down againstthe force of the springs 734. A torsion bar 738 coupled between thelinkage on each side transfers torque from one side to the other tolower each side of the tray together, if either brake pedal 850L, 850Ris pressed. To raise the pedal tray 702, both brake pedals 850L, 850Rare unlatched so that the springs 734 coupled to lift rails can force upand raise the pedal tray 702 above the floor.

The linkage in each side of the pedal system allows the pedal tray to beraised and lowered substantially in parallel to the floor. The linkagein each side of the pedal system includes an upper link 740 and a lowerlink 742 to form a four bar or parallelogram linkage with the lift/guiderail 716R, 716L and the frame of the drive assembly coupled to baseproviding the other links. The upper link 740 pivots with the torsionbar 738 near one end. An opposite end of the upper link 740 pivotsaround a pivotal shaft 746. The lower link 742 pivots near one end abouta pivotal shaft 746. At an opposite end, the lower link 742 pivots abouta pivotal shaft 747. The pivotal shafts 745-746 on each side are coupledto lift/guide rails 716R, 716L of each side of the pedal system 700. Thepivotal shaft 747 may be coupled to the frame 750 of the back driveassembly.

To raise the pedal tray, each side of the linkage is preloaded to beraised up by the springs 734 mounted between the upper link 740 and thebase (or ground) of the surgeon's console. A pick up bracket 748 couplesbetween an end of the gas spring 734 and near a midpoint of the upperlink 740 to couple the spring 734 to the linkage. The preloaded force ofthe spring applied to the linkage tries to keep the rails 716R, 715L(and therefore the foot pedal tray 702) in the raised position. A userpresses down on either brake pedal 850L, 850R to drive the upper link740 down as well, lowering both the linkage and the foot pedal tray 702.If a user pushes sufficiently far enough, either brake pedal 850R, 850Llatches in a down position to hold the linkage and the foot pedal trayin the lowered position so that it can roll over the floor. In the downor lowered position, the rollers 735R and the fool pedal tray can reston the floor without its wheels 715L, 715R resting on the lift/guiderails 716L, 716R.

The pivot point for the upper link 740 near the drive assembly is thetorsion bar 738. The torsion bar extends across to the mirrored linkageassembly in the other side. FIGS. 7B and 7C illustrate the torsion bar738 coupled between the upper links 740 in each side. For this reason,the two linkages and rails in each side move up and down together, evenif only one side's brake pedal 850L or 850R is pressed. When the latchesof the brake pedals 850L, 850R are released, the brake pedals can moveup to their raised position. The springs 734 lift the linkage in eachside up above the floor along with the lift/guide rails. The left andright lift/guide rails 716L, 716R engage the left and right rollers715L, 716R to lift the foot pedal tray 702 to its raised position fortravel.

To move the pedal tray 702 horizontally within the opening 804 along thefloor when lowered or within the guide rails, the pedal system 700further includes a drive mechanism or drive assembly. The drive assemblyof the pedal system 700 includes a tray motor 708 coupled to a frame 750and a scissor arm assembly 720 coupled between the tray motor 708 andthe pedal tray 702. The frame 750 is coupled to the base 801 of thesurgeon's console. The tray motor 708 includes a rotatable shaft coupledto an end of a threaded drive rod 719 to apply a force to the moveablepedal tray 702. The threaded drive rod 719 is rotatably coupled to thebase so that it can rotate in place. One end of a pair of cross arms ofthe scissor arm assembly 720 is threadingly and pivotally coupled to thethreaded drive rod 719. An opposite end of the pair of cross arms of thescissor arm assembly 720 is pivotally coupled to a back side of thepedal tray 702. As the shaft of the tray motor 718 rotates in onedirection, the threaded drive rod 719 rotates in that one direction sothat the scissors arm assembly 720 opens and pushes out the pedal tray702. As the shaft of the tray motor 718 rotates in an oppositedirection, the threaded drive rod 719 rotates in the opposite directionso that the scissors arm assembly 720 closes and pulls in the pedal tray702.

In FIG. 7C, a side perspective view of the pedal system 700 isillustrated. The left vertical switch pedal assembly 706L and the rightvertical switch pedal assembly 706R are pivotally coupled to the leftand right sides of the pedal tray. Each of the left vertical switchpedal 706L and the right vertical switch pedal 706R includes a verticalswitch assembly 722. Each of the pedal assemblies may include a springso as to be spring loaded so they are momentarily closed when pressed onby the user's foot and then spring back to switch open when the usersfoot is removed.

Referring now to FIG. 7E, a cutaway view of the vertical switch assembly722 is illustrated for each of the vertical switch pedals 706L-706R. Thevertical switch assembly 722 includes a rod 730 coupled to the base ofthe pedal tray about which the pedal may pivot to open and close anelectrical switch 732. The vertical switch assembly may further includea spring to push back on the pedal. Assuming a normally open switch,horizontal foot movement against the pedal causes it to pivot about therod 730 to compress the spring and close the electrical switch 732.Releasing the foot pressure on the pedal allows the spring push backout, pivoting the pedal back to a normal position and opening theelectrical switch.

Referring now to FIG. 7D, each of the horizontal pedal assemblies704L-704R and 708L-708R are spring loaded and include a horizontalswitch assembly 724. Each of the horizontal switch assembly 724 includesan electrical switch 725. The horizontal switch assembly may furtherinclude a spring to push back on the pedal. Assuming a normally openswitch, vertical foot movement pressing down against the pedal causesthe spring to compress and the electrical switch 732 to close. Releasingthe foot pressure on the pedal causes the pedal to spring back and openthe electrical switch 732.

Each of the horizontal switch assembly may further include a vibratingdevice 726 to provide haptic/tactile user feedback. Each of thehorizontal switch assembly may further include a sensing device 727 tosense when a foot is over the pedal and ready to press down to close theswitch and generate a control signal to control the robotic surgicalsystem.

Each of the switches 725, 732 activated by the pedal assemblies may beof different switch types, such as a toggle switch (toggling betweenopened and closed switch states), a normally open-momentarily closedswitch, or a normally closed-momentarily open switch. Furthermore, thetype of signal generated by the switch may be translated, encoded, oradapted to a proper voltage to control various types of medicalequipment.

Ergonomic Control

Referring now FIG. 9, a perspective view of the person of the workspace316 of the surgeon's console 150, 150A is illustrated. As mentionedpreviously, the pedal tray 702 of the pedal system at the surgeon'sconsole moves so that the pedal positions are comfortably reached by thesurgeon. For each surgeon, a desired saved pedal position can berecalled and the pedal tray can be automatically adjusted to custom fita size of the surgeon.

To initially control (and override the automated adjustments) themovement of the pedal tray 702 of the pedal system 700, as well as otherergonomic positions, the surgeon's console 150, 150A includes anergonomic control panel 911 next to the arm wrest 314. The ergonomiccontrol panel 911 is better illustrated in FIG. 11A.

Referring now to FIG. 11A, a magnified perspective view of the ergonomiccontrol panel 911 is illustrated. The ergonomic control panel 911includes a pedal tray adjustment toggle switch 1110, an arm-rest heightadjustment toggle switch 1112, a display height adjustment toggle switch1114, and a display angle adjustment toggle switch 1116. Each of thesetoggle switches may be moved up or down to energize a motor to changeposition settings. For example, the pedal tray adjustment toggle switch1110 when pushed upward may turn on an electric motor to adjust thepedal tray inward. When the pedal tray adjustment toggle switch 1110 ispushed downward it may turn on the electric motor to adjust the pedaltray outward.

The switches 1110, 1112, 1114, and 1116 of the control panel 911 arepositioned into a console map 1101 of the panel 911 with icons of theconsole in the background so that it is clear what function each has.The pedal tray adjustment toggle switch 1110 that moves the pedal tray702 is located near feet icon of the console map to indicate positionadjustments of the pedals. The arm-rest height adjustment toggle switch1112 is located on an arm-rest icon to indicate height adjustments ofthe arm-rest. The display height adjustment toggle switch 1114 islocated on a monitor icon to indicate height adjustments of the stereodisplay. The display angle adjustment toggle switch 1116 is located onan axis icon near the monitor icon to indicate angle adjustments of themonitor.

The ergonomic settings of the surgeon's console set by each of thetoggle switches may be saved into a user account for each respectiveuser. The information regarding the account may be stored in a storagedevice of the computer 151. Each user account, including the savedergonomic adjustments, may be secured with a login identification ID andoptionally a password.

Referring now back to FIG. 9, the surgeon's console 150, 105A furtherincludes a liquid crystal display (LCD) touchpad screen 912 mounted inthe arm-rest 314 to access the user accounts. The LCD touchpad screen912 is located near the center of the arm-rest 314 between positionswhere a surgeon's arms may wrest so that it may be viewable duringsurgery and avoid extraneous readings from the arms touching the touchscreen.

Referring now to FIG. 11B, a top view of the liquid crystal display(LCD) touchpad screen 912 is illustrated. The saved ergonomic settingfor a user may be recalled by logging into the robotic surgical systemthrough the use of the touchpad screen 912. A surgeon (user or operatorO) may enter or select his or her login ID and optionally enter apassword by means of a touch sensitive keypad 1122 of the touchpadscreen 912. After a successful login to a user account, the systemrecalls the ergonomic settings, including the pedal tray adjustmentsetting, the arm-rest height adjustment setting, the display heightsetting, and display angle setting. The system then automaticallyreadjusts the positions of the pedal tray, the arm-rest and the stereodisplay to those saved settings. For example, the position setting ofthe pedal tray 702 with respect to the console 150, 150A is recalledfrom the storage device and automatically adjusted to that setting thatis associated with the user account. If any of the settings areundesirable to the user, he can reset the adjustments for current useand then save the new settings upon logging out of the system for laterrecall.

Referring now back to FIG. 9, the position of the master controllers905L and 905R may be adjusted to a desired position by a user withoutmoving/controlling the robotic arms and robotic surgical tools. This isby selectively disengaging the master controllers from the slavecontrolled arms and tools, sometimes referred to as clutching. That is,the master controllers 905L and 905R can be clutched from the roboticarms and surgical tools to adjust their positions.

Previously, this may have been performed by pressing a pedal or acombination of pedals to clutch the master controllers. In this case,both of the master controllers were clutched. However, there are timeswhen only one master controller need be clutched. This may be the case,for example, if only one needs to be repositioned. Alternatively, one ofthe master control arms may be used to control a mouse pointer(sometimes referred to as a masters as mice mode) to select items froman on-screen menu system in the graphical user interface for furthercontrol of the robotic surgical system and/or its surgical tools.

To further control different modes and independently adjust thepositioning of the left master controller and the right mastercontroller, the left master grip 925L and the right master grip 925R mayeach may include one or more switches.

Referring now FIG. 10, a magnified perspective view of the right mastergrip 925R and right control input wrist 952R of the right mastercontroller is illustrated. The right master grip 925R and right controlinput wrist 952R is also representative of the left master grip and leftmaster wrist portion of the left master controller.

Each of the left master grip 925L and the right master grip 925Rincludes a front side switch 1010F and a back side switch 1010B that canbe controlled by fingers of the user (surgeon, operator O). The frontand back side switches on each master controller may be identical butread independently by the system. However, software may be programmed toperform the same action in response to activation of either the front orback side switch. This permits a user to grasp the masters in differentways and still control a controllable feature with either of the frontside switch 1010F and the back side switch 1010B of a respective mastergrip. Collectively, the front side switch 1010F and the back side switch1010B of both master controllers may be referred to herein as a switch1010.

In one embodiment of the invention, the switch 1010 is a slideableswitch that can move from a closed position to an open position and backagain. The switch slides along a tubular support structure 351 of themaster control input grip 925R, 925L. In one embodiment of theinvention, the switch 1010 is not spring loaded so that it remains inwhatever position is chosen by the user. In another embodiment of theinvention, the switch 1010 is spring loaded, requiring the user tomaintain pressure on the switch in one of the positions (open orclosed), otherwise having it slide back to a normal position (closed oropen) when the sliding pressure on the switch is released.

The switch 1010 of each master grip can be programmed to independentlyclutch each of the left master controller and right master controllerfrom the respective slave controlled elements (left robotic arm/tool andright robotic arm/tool) of the patient side cart 152. After clutchingthe left or right master controller, the clutched master controller canbe repositioned within the workspace so as to be in a better position tocontrol the robotic surgical tool when the master controller isre-engaged (un-clutched) to control the respective slave controlledelements of the patient side cart 152. The master controller can bere-engaged by switching the switch back to normal control mode.

When clutched by the switch 1010, the clutched master controller alsomay be used as a multi-dimensional computer mouse moving in multipledimensions over a multi-dimensional space displayed by a stereo viewer.In one case, the clutched master controller is used as athree-dimensional computer mouse moving in three dimensions over athree-dimensional space displayed by the stereo viewer. The userinterface in that case may have certain three dimensional aspects suchas described in U.S. patent application Ser. No. 12/189,615 entitledINTERACTIVE USER INTERFACES FOR ROBOTIC MINIMALLY INVASIVE SURGICALSYSTEMS, filed on Aug. 11, 2008, by Simon DiMaio, for example, andincorporated herein by reference.

While the switches 1010 in each master controller can independentlyclutch each master controller, a clutch pedal or clutch pedal sequencemay still be provided/used to jointly clutch both master controllerstogether, such as when the surgical procedure may be completed, forexample. Furthermore, while one of the switches 1010 may be used toenter a masters-as-mice mode, the masters as mice mode may alternativelybe entered by stepping on a foot pedal or by pressing a button on atouchpad interface, for example, Moreover, the functionality of theswitches 1010 may be expanded. For example, the switches 1010 may beselectively used to activate an energy device with a user's hand, swapcontrol from one slave arm to another, or activate a new feature.Furthermore, front and back switches 1010E-1010B may be programmed toperform different actions in response to varying contexts or userinterface modes.

Electro Surgical Instruments

Robotic electrosurgical tools can be mounted to one or more of therobotic arms of the patient side cart of the robotic surgical system.Details of robotic electrosurgical tools are described in U.S. patentNos. with filing dates and named inventor as follows U.S. Pat. No.6,840,938, Dec. 21, 2001, Morley et al.; and U.S. Pat. No. 6,994,708,Apr. 18, 2002, Scott Manzo; and U.S. Pat. No. 7,320,700, Jan. 22, 2008,Cooper et al.; and U.S. Pat. No. 7,367,973, May 6, 2008, Manzo et al.;and Ser. No. 11/238,794, Sep. 28, 2005, Scott Manzo; and Ser. No.11/535,426, Sep. 26, 2006, Manzo et al.; all of which are incorporatedherein by reference.

Generally, robotic electrosurgical instruments and systems can be usedfor electrosurgical treatment of tissue during minimally invasiverobotic surgical procedures. The electrosurgical instruments are capableof treating tissue with heat produced by electrical energy whilecutting, shearing, grasping, engaging, or contacting treatment tissue.For example, an electrocautery instrument may be used to apply anelectrical current to tissue in a surgical site so as to burn or sealruptured blood vessels.

Electrosurgical instruments may apply a high-frequency alternatingcurrent to surrounding tissue, thereby to cause the tissue temperatureto rise to the point where the tissue is cut or coagulates.Alternatively, electrosurgical instruments may apply heat to tissue bymeans of electrically generated heat inside the instrument. Regardless,electrosurgical treatments may further reduce bleeding of tissue bycauterizing tissue and coagulating blood, or achieve various otherdesired effects on the treatment tissue such as sealing tissue together.The electrosurgical treatment is carried out in a safe and effectivemanner that incorporates a variety of safety features to prevent currentleakage to non-target tissue so as to reduce collateral tissue damage,unwanted burning, or the like.

For electrosurgical instruments, a range of supply settings may be usedfor cutting, coagulation and the like. An exemplary power supply cancreate a power output up to approximately one-hundred-twenty (120) watts(W) for cauterizing the target tissue. The voltage used with a bipolarrobotic cauterizer tool is generally between zero (0) volts (V) and onethousand (1000) V peak-to peak, and preferably between one-hundred (100)V and five-hundred (500) V. As long as the jaws and electrodes are bothin good contact with the tissue intended to be cauterized and/or cut,there is much less chance of voltage from the electrodes arcing to otherconductive components on the instrument (e.g., the wrist, shaft, orpulleys). It should be appreciated, however, that the voltage setting ofthe electrosurgical power generator will vary depending on the specificdimensions of the electrodes, the tissue to be treated, and the like.

Referring now to FIG. 2A, robotic electrosurgical instruments 101A, 101Band endoscopic camera 101C are mounted to the robotic surgical arms 153.Each of the robotic electrosurgical instruments typically include anelongated shaft, with proximal end coupled to a housing and a distal endhaving one, two, or more end effectors 212. The end effectors 212 aregenerally mounted on wrist-like mechanisms pivotally coupled to thedistal ends of the shafts, for enabling the instruments to perform oneor more surgical tasks. Generally, the elongated shafts of surgicalinstruments allow the end effectors 212 to be inserted through entryports in a patient's body so as to access the internal surgical site.Movement of the end effectors 212 and the electrosurgical instruments isgenerally controlled via master controls at the master console 150,150A.

Electrosurgical instruments may be categorized as being monopolar orbipolar. For either electrosurgical tool type, the energization of oneor both electrodes in the end effectors 212 is controllable by thesurgeon operating the surgeon's console 150, 150A.

For a monopolar electrosurgical tool, the patient is grounded and avoltage is supplied to tissue through an electrode coupled to an endeffector. In the monopolar tool, an electrically conductive cable withinthe tool extends from one plug 474A of the housing 401 to the electrodeat the end effector. An external cable with first and second ends,releaseably plugs into the plug 474A at the first end and to anappropriate electrical generating source unit (ESU) at the second end.The plug 474A of the tool may typically be a conventional banana-typeplug to receive a banana-type connector.

For a bipolar electrosurgical tool, a voltage is supplied to tissuethrough the use of two electrodes, a first electrode coupled to a firstend effector and a second electrode coupled to a second end effector.The two electrodes on the end effectors of the bipolar tool whenenergized are set at two different electrical potentials and preferablydo not come in contact with each other when energy is applied.

In a bipolar tool, a pair of electrically conductive cables within thetool extend from two plugs 474A, 474B of the housing 401 to the twoelectrodes at the respective two end effectors. A pair of externalcables with first and second ends, releaseably plug into the plugs 474A,474B at the first end and to a pair of plugs of an appropriateelectrical generating source unit (ESU) at the second end. The plugs474A, 474B of the tool may typically be a conventional banana-type plugto receive a banana-type connector.

It will be appreciated that a number of elements of the tool are formedof a non-conductive or insulative material, such as, ULTEM, electricalgrade fiberglass/vinyl ester composite material, or a liquid crystalpolyester VECTRAN or coated with a non-conductive or insulativematerial, e.g., a nylon or parylene, such as Nylon-11 or Parylene C. Forexample, the housing 401 is typically of a non-conductive plasticsmaterial. As another example, the shaft 404 is typically made entirelyfrom a nonconductive material, or at least sheathed in such a material,to insulate the shaft from the patient, in particular in the region ofthe port of entry coincident with the center of rotation. Onenonconductive material for the shaft comprises an electrical gradefiberglass/vinyl ester composite material. Alternatively, a shaft ofstainless steel or carbon fiber may be coated with, e.g., a nylon orparylene, such as Nylon-11 or Parylene C. It has been found that theelectrode at the end effectors should be insulated from the rest of theinstrument 400 so as to inhibit current leakage from the electrode toother elements in the instrument 400.

While electrosurgical tools and systems are described herein withrespect to the embodiments of the invention, the principles, methods,techniques, systems and apparatus of the invention may be applicable toother types of tools and systems.

User Interfaces for Electrosurgical Systems

An electrosurgical tool may be mounted to any one or more of the roboticarms of a surgical system and manipulated by a surgeon from thesurgeon's console. In some cases, two electrosurgical tools are mountedto a left robotic arm and a right robotic arm. Previously, keeping trackof which tool in which side or hand was to be electrically activated wasleft to memory, particularly when the same type of electro surgical toolwas mounted to each arm. Different foot pedals for different types ofelectrosurgical tools may have provided some guidance on the handednessof the electrosurgical tools.

An enhanced user interface for electrosurgical systems is disclosedherein to allow control swapping between robotic surgical tools, such asa plurality of robotic electro-surgical tools, and to provide anindication to a surgeon as to which side and tools are ready to provideelectrosurgical energy and which side is actually activated (firing) todeliver electrosurgical energy to the tissue.

During initial testing of a graphical user interface for a newintegrated energy management system, a bright blue colored border wasplaced around an edge of either the left or right side of the surgeon'sdisplay to indicate if the left or right tool was receiving energy. Itwas discovered that surgeons sometimes do not notice the bright bluecolored border at the edge of a display, perhaps because they arefocusing on the tissue of interest in the center of the display. As aresult, surgeons sometimes do not realize which electrosurgical tool isready to be energized (activated), or that they are energizing(activating) an electrosurgical tool when they did not intend to.Alternatively, a surgeon may not realize they are energizing(activating) an electrosurgical tool on the wrong side, opposite thedesired side.

The enhanced user interface for electrosurgical systems disclosed hereincan provide further information to the surgeon so they know which sideand tool is ready to provide electrosurgical energy and which side isactually activated (firing) to provide electrosurgical energy. There arethree possible ways that may be used alone or in combinations of two ormore to provide an enhanced user interface for electrosurgical systemsthat target the human senses: an enhanced visual display, an enhancedaudio output, and/or haptic output or feedback.

Enhanced Graphical User Interface for Electro Surgical Systems

One possible method of further informing a surgeon of theelectrosurgical tools is to provide an enhanced visual display through agraphical user interface (GUI) that is displayed to the surgeon at thesurgeon's console. A graphical user interface may provide a persistentvisual indication that is easier to notice on the periphery of thescreen. The GUI may alternatively provide movement in the peripheralvision of the user which tends to be highly effective at attracting userattention. For example, a moving barber pole pattern that zips up anddown at the side of the screen, or a different color or light on theside corresponding to the “hot hand” are examples of such displays. TheGUI may provide persistent, and/or a momentary indication of anelectrosurgical instrument having the capability of being activated.This indication may be associated with or overlaid upon the image of theinstrument itself. The GUI may also provide a momentary visualindication whenever energy is activated. For example, the border aroundthe left or right side could flash or change color when energy isactivated to give the user a visual change when energy is activated.

In FIG. 4, a graphical user interface may be displayed around or nearedges of the display like a border. In a stereoscopic display 312, aleft user interface border 461L and a right user interface border 461Rmay be overlaid (fused) onto the video images of a surgical site asshown displayed on display devices 452L, 452R.

Referring now to FIGS. 5A-5C, an electrosurgical graphical userinterface (GUI) 502 for a surgical system is shown overlaid onto (fusedtogether with) video images 501 of a surgical site in a display 500.Note that the left robotic surgical tool image 101L and the rightrobotic surgical tool image 101R in FIGS. 5A-5C are exemplary images andmay not necessarily be representative of electrosurgical tools nor thespecific types of robotic surgical tools indicated by the GUI 502. TheGUI 502 may be fused together and overlaid onto the video images 501 bya graphics generator 322. The display 500 shown in FIGS. 5A-5C mayrepresent the left display 452L, the right display 452R, or what thesurgeon sees when viewing the stereoscopic display 312.

The GUI 502 may be overlaid as a user interface border on the videoimages 501 including a left colored border 502L and a right coloredborder 502R. The GUI 502 further includes a left tool number icon 504L,a right tool number icon 504R, a swap tool number icon 504S, a left tooltype text icon 505L, a right tool type text icon 505R, a swap icon 506,and a swap tool type text icon 509. The user interface 502 furtherincludes master control icons 510 and an energy ball icon 508. The userinterface 502 may further include a camera orientation icon 507. Thecamera orientation icon 507 may be used to indicate the orientation ofthe video or camera images 501 captured by the endoscopic camera.

In FIG. 5A, the left colored half frame or left colored border 502L andthe right colored half frame or the right colored border 502R may usedifferent colors to indicate either an inactive energy tool type or anactive energy tool type controlled by the left hand and right handrespectively. For example, the left colored border 502L may have a color(e.g., a brown color) to indicate an inactive energy tool typecontrolled by the left hand, foot and/or side of the surgeon's consolethat does not have the capability to couple energy into tissue. Theright colored border 502R may have a color (e.g., a blue color) toindicate an active energy tool type that is capable of coupling energyinto tissue to cut or cauterize and is controlled by the right hand,foot and/or side of the surgeon's console. If the energy control ismultiplexed to the opposite side, the right colored border 502R maychange color (e.g., to a brown color) such as illustrated in FIG. 5C toindicate that the right hand tool will not be activated to couple energyinto tissue, even though the tool has that capability.

In FIG. 5A, the GUI 502 illustrates that the right-hand instrument is anenergy instrument (e.g., a permanent spatula cautery tool) and is readyto fire or apply energy to tissue. The color of the right half-frame502R on the right-hand side informs the user that when one of the energypedals is stepped on, the instrument currently controlled by the user'sright hand will become energized to apply energy into tissue.

The robotic surgical tool mounted to robotic arms of the patient sidecart may both be electrosurgical tools. In which case, it may bedesirable to swap energy control of the surgeon's console between toolscontrolled by the left and right hands. This may be referred to asswapping the handedness (or sidedness) of the energy control of thesurgeon's console, even though the same foot and foot pedals are used tocontrol the electrosurgical tool.

The energy ball icon 508 indicates the handedness of the energy controlof the surgeon's console. If the handedness of the energy control isswapped, the energy ball icon 508 is accordingly swapped between theright side (near center) and the left side (near center) of the displayto indicate which tool may be energy controlled by controls (e.g., thefoot pedals) at the surgeon's console.

The background color and symbol overlaid onto the energy ball may beused to indicate either an inactive energy tool type or an active energytool type controlled by the respective handedness. This is because thehandedness of the energy control may be swapped to a tool that isincapable of delivering energy to tissue. In FIG. 5A, the energy ballicon 508 is illustrated with a blue background and a yellow lightningbolt, for example, to indicate an active energy handedness associatedwith the right tool. In FIG. 5C, an inactive energy ball icon 508″ isillustrated with a grey background and a grey strike-through lightningbolt, for example, to indicate an inactive energy handedness associatedwith the left tool in contrast with the active energy ball icon 508.

The left tool icon number 504L and the right tool icon number 504R mayillustrate the number of the tools that are mounted to robotic arms ofthe patient side care that are capable of being controlled by thesurgeon's console. The colors of the left tool icon number 504L and theright tool icon number 504R may be colored similar to the left coloredborder 502L and the right colored border 502R to further indicate eitheran inactive energy tool type or an active energy tool type controlled bythe left hand and right hand respectively. The swap tool icon number504S indicates the tool number to which a swap of control may be made bythe surgeon's console. The swap tool icon number 504S may be colored toindicate that its associated robotic tool is not presently controlled bythe surgeon's console. The swap icon 506 indicates whether or not analternate tool may be controlled by a master grip of the surgeon'sconsole and that a swap may be made. The position of the swap icon nearthe right or left side border of the user interface indicates thehandedness of the swapped tool. The swap tool text icon 509 in the rightside of the GUI indicates the alternate tool that may be controlled withthe right master grip of the surgeon's console.

If a tool swap is initiated by a surgeon, the swap tool icon number 504Sand the right tool icon 504R may swap positions and colors about theswap icon 506 to indicate the associated robotic tool which is presentlycontrolled and which tool is in not presently controlled (also referredto as being idle) by the right master grip of the surgeon's console. Theright tool type text icon 505R and the swap tool text icon 509 may alsoswap positions and colors to further indicate the active roboticsurgical tool which is presently controlled by the surgeon's right handon the right master grip of the surgeon's console in contrast to theidle robotic surgical tool. Generally, robotic surgical toolscontrollable by the right master controller are represented in the righthalf of the user interface and robotic surgical tools controllable bythe left master controller are represented in the left half of the userinterface.

The camera orientation icon 507, near a top center of the GUI 502,illustrates the orientation of the endoscopic camera with respect to areference, the patient side cart. This allows the surgeon to understandthe orientation of the video images with respect to the patient sidecart of the robotic surgical system. This may help orient the patient'sbody as well so that the surgeon is better equipped to understand theorientation of organs and tissue therein.

The master control icons 510 illustrate a mapping of the physicalcontrols at the surgeon's console. The master control icons 510 includea pedal map and a master grip map. The master control icons 510 are bestillustrated in FIG. 6A-6B and are now described in further detail.

Referring now to FIGS. 6A-6B, a magnified view of the master controlicons 510 is illustrated, including pedal icons 600 and master controlgrip icons 620. The pedal icons 600 include a lower left pedal icon604L, a lower right pedal icon 604R, a left switch pedal icon 606L, aright switch pedal icon 606R, an upper left pedal icon 608L, and anupper right pedal icon 608R. The pedal icons 600 may further include anupper pedal function icon 611U and a lower pedal function icon 611L. Themaster control grip icons 620 include a left grip control icon 625L anda right grip control icon 625R.

The pedal icons 600 of the master control icons 510 are displayed in apedal map (position and orientation) that matches the position andorientation of the respective controllable foot pedals (e.g., see FIG.7A). The pedal map reminds the surgeon which foot (left or right) andfoot position (top, bottom, or side position) controls the controllablefoot pedals. In this manner, to increase efficiency, a surgeon may notneed not move his head away from the stereoscopic display and view hisfeet over the controllable foot pedals to control them.

The master control grip icons 620 are displayed in a master grip map, aleft master control grip icon 625L representing a status of the leftmaster grip and a right master control grip icon 625R representing astatus of the right master grip of the surgeons' console. The mastergrip map of the master control grip icons 620 may also remind thesurgeon where to position his hands to grab the master grips. The statusindication provided by the master control grip icons may indicated whichmaster control grip may be clutched to reposition a hand in space orcontrol some other controllable feature at the surgeon's console.

During electrosurgical procedures, the controllable foot pedals (e.g.,see FIG. 7A) may have assigned functionality. The pedal icons mayindicate the assigned functionality of the controllable foot pedals. Asillustrated in FIG. 6A, an upper pedal function icon 611U near the upperright pedal icon 606R indicates Mono Cut (monopolar cutting) associatedwith the upper right pedal while the lower pedal function icon 611L nearthe lower right pedal icon 604R indicates Mono Coag (monopolarcoagulation) associated with the lower right pedal. The pedal map ofpedal icons may further include a pedal outline or halo 612 around oneof the pedal icons to indicate the activation of pedal to the user andapplication of energy to tissue.

The pedal map of pedal icons may further include a left function icon610L near the left switch pedal icon 606L associated with thefunctionality of the left horizontal switch pedal and a right functionicon 610R near the left switch pedal icon 606L associated with thefunctionality of the right horizontal switch pedal. In FIG. 6A, a swaptool icon 610L in the shape of the letter “S” with arrowheads isillustrated near the left switch pedal icon 606L. An energy swap icon610R in the shape of a lightning bolt with arrowheads nearby isillustrated near the right switch pedal icon 606R to illustrate theenergy swapping functionality of the right horizontal switch pedal.

The left grip control icon 625L and/or the right grip control icon 625Rmay inform the user of active control of the respective left and/orright tools at the surgeon's console. In response to a left masterswitch or a right master switch, the left and/or right tools may beclutched from control of the robotic arms and surgical tools theycontrol. Either or both icons may indicate inactive tool control—clutchof the master grips at the surgeon's console from tool control—so thatthe left master grip and/or the right master grip may be used to controlother features of the robotic surgical system without movement of asurgical tool.

As discussed previously, the energy ball icon 508 indicates thehandedness of the energy control of the surgeon's console. In FIG. 6A,the right-handed energy ball 508 is illustrated to the right of themaster control icons 510. In this case, the electrosurgical tool in theright hand is to be energized by one or more pedals at the surgeon'sconsole. The magnified view of the pedal map shows which pedal iscurrently being activated by outlining or haloing the Mono Coag pedalicon 604R with a colored halo or outline 612 (e.g., orange color) thatdiffers from the color fill (e.g., blue) of the pedal icon. The coloredhalo or outline 612 activation indicator reinforces the informationprovided to the user as which pedal is being activated.

The pedal map of pedal icons, including the colored halo or outline 612indicator, helps to teach users the foot pedal layout without requiringthem to remove their head from the 3D stereoscopic display to actuallylook at the position of their feet over the pedals. Unlike the coloredside-bar or activated energy strip bar or border 522, the pedal map ofpedal icons and the colored halo or outline 612 activation indicator isplaced more towards the center of the surgeon's vision so as to not relyon peripheral vision alone. If the surgeon steps on the Mono Coag energypedal, the Mono Coag pedal icon 604R is outlined or haloed by thecolored halo or outline 612 activation indicator as best shown in FIG.6A. If the surgeon steps on the Mono Cut energy pedal instead, the MonoCut pedal icon 608R would be outlined or haloed by the colored halo oroutline 612 activation indicator instead of the Mono Coag pedal icon604R. In either case, the colored side-bar or activated energy stripborder 522 is displayed in an active side such as shown in FIG. 5B wheneither energy pedal is selected by the user and energy is ready to bedelivered to an instrument. A terracing of the upper and lower footpedals deters the surgeon from concurrently pressing both upper andlower foot pedals.

To switch the energy activation from a tool in one hand to a tool in theopposite hand, an energy swap pedal (e.g., pedal 706R-a horizontalswitch pedal) is selected by the surgeon's foot to toggle the control.The energy ball 508 moves from the right side to the left side in theGUI near the pedal map of the pedal icons 600. FIG. 6B illustrates anenergy ball icon 508″ positioned in the GUI to the left of the mastercontrol icons.

In FIG. 6B, the left tool is a bipolar electrosurgical tool such thatthe lower right foot pedal (e.g., pedal 704R) is activate as indicatedby the lower pedal function 611L′. With a bipolar electrosurgical tool,only one foot pedal may be used to control energy. Thus, the upper pedalfunction icon 611U′ indicates no functionality associated with the upperright foot pedal—an inactive foot pedal. The upper pedal icon 608R′ mayfurther indicate its inactive status by being filled with an inactivecolor (e.g., a grey color fill) and/or displayed with a strike-through.

The pedal icons themselves may have different colors or overlaidillustrations to further indicate their functionality to the surgeon.The upper pedal icon 608L may include an illustration of crossed doubledheaded arrows (an up and down arrow crossing over a left and rightarrow) to indicate that it allows free movement of the mastermanipulators, temporarily decoupling them from the surgical instruments.The lower left pedal icon 604L has an illustration of a camera icon tofurther inform the user that it allows the user to control the motion ofthe endoscopic camera. The lower right pedal icon 604R may have adifferent color fill (e.g., blue colored fill) than the upper rightpedal icon 608R (e.g., yellow colored fill) to illustrate differentcontrols to apply different levels of energy to a mono-polarelectrosurgical tool. The lower right pedal icon 604R may have adifferent color fill to illustrate a single activate control of abipolar electro surgical tool while the upper right pedal icon 608R maynot be present or provide an indication of being inactive.

Referring now to FIG. 5B, assume the surgeon activated the energy pedal704R to begin energizing the electrosurgical tool on the right handside. The pedal halo 612 surrounds the lower right pedal icon 604R toillustrate activation by the pedal 704R. An activated energy stripborder 522 with a different color than the right side border 502R may beoverlaid onto the right hand side during electrosurgical toolactivation. The activated energy strip border 522 further illustratesenergy activation of the right side tool and not the left side tool. Thesurgeon can readily see the activated energy strip border 522 because itchanges the right side border 502R when the foot pedal is depressed toactivate the electrosurgical generating unit (ESU) to couple energy intotissue. If instead the left hand side is to be activated, the activatedenergy strip border 522 with a different color than the left side border502L may be overlaid onto the left hand side during electro surgicaltool activation.

In FIG. 5B, the pedal halo or outline 612 surrounding the lower rightpedal icon 604R illustrates activating monopolar coagulation energy. Acolored bar (the activated energy strip border 522) on the right side ofthe display serves as a change in the user's peripheral vision. Thischange in the user's peripheral vision reinforces informing the user ofwhich side energy is being applied, without requiring the user toactively look at the indicator (the colored bar or activated energystrip border 522). As mentioned previously, instead of the activatedenergy strip border with a different color, a graphical barber-pole thatillustrates movement may be used in the GUI to inform a user.Alternatively, a graphical spinning wheel may be displayed in a side orcorner of the GUI. In yet another embodiment, a bright pulsing light mayslide or bounce up and down along a side portion of the GUI border.Alternatively, a series of graphical light bars may light up whenelectrosurgical energy is to be supplied to a tool. Collectively, thesegraphical display devices may be referred to as an activated energyindicator. While the graphical images of an activated energy indicatorhave been described as being in the periphery and overlaid onto aportion of the GUI border, they may be located elsewhere. For example,an activated energy indicator may be overlaid on top of and/orautomatically positioned near the images of the robotic surgical toolsor instruments. The activated energy indicator in this case may trackthe tools or instruments as they move around the surgical site.

A surgeon may toggle left/right hand side control with an “energy swap”pedal whenever desired, independently of whether there is anelectrosurgical capable tool in either hand. However, the active energycolor border (e.g., blue) is displayed in the right and/or left userinterface borders 502L-502R only when all of the necessary conditionsfor enabling energy activation are met. Whenever the energy swap pedalis pressed, the side (left or right) capable of delivering energychanges. This is represented in the GUI by the little energy ball 508,508″ near the lower border (frame-half) portion in either side of thedisplay. The energy ball swaps sides each time the energy swap pedal ispressed.

When the energy ball is swapped to a side that has an electrosurgicalcapable tool mounted to its robotic arm and the appropriateelectrosurgical generator for that type of tool is also detected, theGUI displays an active border (e.g., bright or blue in color) on thesame side as the energy ball, an active energy ball 508 with an activebackground or fill color (e.g., bright or blue in color) and an activesymbol such as a yellow colored lightning bolt, and an inactive border(e.g., dark or brown in color) on the opposite side of the energy ball.

In FIG. 5A, by use of the energy swap pedal, the surgeon has assignedenergy to the right hand side, denoted by the active energy ball 508 tothe right of the foot pedal map of the master control icons 510 next tothe right tool text icon 505R with the exemplary text Permanent SpatulaCautery.

If a user swaps the energy ball when there is an energy-capableinstrument on both hands, the active colored half-frame jumps from leftto right along with the active energy ball, thus immediately visuallynotifying a user that a change has taken place, and providing a constantindication as to which side (left or right) will be activated when theenergy pedal(s) are pressed. If a user fails to pay attention to thebright blue frame and steps on one of the energy pedals when the “wrong”side is selected, the appearance of the activated energy strip bar 522(e.g., a bright orange bar) serves as a visual reminder that is verystrong in the peripheral vision as to which side is being activated.This visual feedback to the surgeon/user can help mitigate mistakes andreinforce in the user's minds which side energy has been swapped to.

When an energy ball 508 is swapped to a side that does not have anelectrosurgical capable tool, or if the system does not detect thenecessary electrosurgical generator (ESU) for an electrosurgical toolcurrently controlled by that side, the GUI display an inactive bordercolor (e.g., dark or brown color) on the opposite side of the energyball, inactive border color (e.g., dark or brown color) on the same sideas the energy ball, and an inactive energy ball 508″ with an inactivebackground or fill color (e.g., dark or gray color) and an inactivesymbol such as a faint outline of a lightning bolt with a diagonal slashthrough it (e.g., a diagonal strike-through).

Referring now to FIG. 5C, the energy swap pedal 706R is selected to swapenergy control to the left hand side tool. The energy ball icon 508″ islocated to the left of the master control icons 510 to indicate theenergy foot pedals are used to control a left hand side tool. However,the robotic surgical tool controlled by the left hand side is notcapable of providing energy to tissue—it is not an electrosurgical tool.While the energy ball icon 508″ is located on the left hand side of theuser interface, it has an inactive color fill (e.g., a gray color fill)and a strike-through lightning bolt to inform the user that energy willnot be provided to the tissue in this case, even if an energy pedal isselected. Further, a color (e.g., brown) of the left side border 502Land the right hand side border 502R′ indicate a lack of energycapability in both the left and right hand sides. That is, in this caseenergy will not be provided to the tissue through either the left handor right hand robotic surgical tools.

In FIG. 5C, the surgeon has enabled the left hand side to deliverenergy. However, the tool mounted to the robotic arm under control ofthe left hand side is a Cadiere Forceps which is not an energy-capabletool. Although the energy ball 508″ has been moved to the left handside, there is now no capacity to deliver energy to tissue. Thus, boththe left and right half frames or borders of the GUI are colored asbeing inactive (e.g. dark or brown color). The GUI would be displayedthe same if there were an energy-capable tool in the left hand and aninappropriate electrosurgical energy generating device were connected tothe system. The indication to the surgeon that no energy is available ismost quickly evident by the lack of an active half frame (e.g., blue orbright color) on the left or right of the screen, and can be seen bylooking at the type of energy ball being displayed as well.

A user may decide to swap the tool that is being mechanically controlledby the left or right master grips and associated left and right hands byselecting a tool swap pedal (e.g., pedal 706L shown in FIG. 7A).

The visual feedback provided by the GUI is somewhat master-centric inthe sense that it informs the user as to whether the instrumentcontrolled by his left hand or right hand will fire (energize or becomeactive) when one of the energy pedals is pressed. In the case of threeor more instruments, this paradigm doesn't need to change. At any onetime, only one instrument is actively assigned to be controlled by theleft master grip/left hand and only one instrument is actively assignedto be controlled by the right master grip/right hand. The activelycontrolled instruments have active tool text icons 505R, 505L displayedin the graphical user interface that are large with bold text. Incontrast, idle instruments that are waiting to be controlled by a swapcontrol signal from the user have a swap tool text icon 509.

The swap tab or swap tool text icon 509 off to the side in the corner ofthe GUI is smaller in size and color filled to indicate inactivecontrol. (e.g., dark or gray or brown colored tab). The swap tool texticon 509 includes text indicating the type of instrument (e.g., a doublefenestrated grasper) that is mounted to another one of the right sidearms. The swap tool text icon or swap tab in the right side indicatesthat the third instrument is associated with the right master. However,the third instrument is currently not the active instrument in the rightside as shown in the Figures. The first instrument indicated by theright tool type text tab or icon 505R is active. If a user activates thetool swap or arm swap pedal 706L, his right master would ceasecontrolling the Permanent Spatula Cautery instrument, and would startcontrolling the Double Fenestrated Grasper instrument. This change maybe reflected in the GUI by swapping the positions of the two instrumentname tabs or icons 505R and 509 or 505L and 509. The tool swap pedal706L may generate a tool swap signal to swap the kinematic control bythe right master grip between a pair of robotic surgical tools. With achange of context, it may also be used to generate a console swap signalto swap some control over the remote controlled equipment and roboticsurgical tools between a pair of surgeon control consoles.

Enhanced Haptic User Interface for Electro Surgical Systems

In addition or alternatively to visual feedback provided by the GUI,tactile feedback may be used to indicate the energy status of anelectrosurgical tool to a user. A user may be informed as to whicheverelectrosurgical instrument is ready to be energized (sometimes referredto as being hot or ready to fire) by one or more types of vibration atpoints where a user makes contact with the surgeon's consoler 150, 150A.For example, the haptic feedback output may be coupled to the surgeon'shands, feet, and/or arms.

A vibration representing the haptic or tactile feedback may be achievedby use of a dedicated (e.g. vibrotactile) device or by making furtheruse of one or more pre-existing devices in the surgeon's console, suchas the powered axis of the input control wrists of the mastercontrollers. In either case, a haptic/tactile feedback generator 321 inthe surgeon's console 150, 150A, such as illustrated in FIG. 3A, may beused to generate and control the haptic feedback. The haptic/tactilefeedback generator 321 may generate a left feedback control signal to becoupled to left side vibrating mechanisms/devices and a right feedbackcontrol signal to be coupled to right side vibrating mechanisms/devices.

Haptic feedback may be provided to the surgeon's hands. In FIG. 3C, eachmaster grip control input 325 (left and right side) may include avibrating feedback mechanism 399 to provide tactile feedback.Alternatively, the roll motor 370 a of the input control wrist or anelectro magnet 396 of the master grip control input may be controlled bythe haptic feedback generator 321 in the surgeon's console to generate avibration in the left or right master grip controller 325.

Haptic feedback may be provided to the surgeon's feet. In FIG. 7D, eachof the horizontal pedal assemblies 704L-704R may further include avibrating feedback mechanism 726. In FIG. 7E, each of the vertical pedalassemblies 706L-706R may further include a vibrating feedback mechanism736.

Haptic feedback may be provided to the surgeon's arms. In FIG. 9, thearm-rest 314 in the Surgeon's console may include a left vibratingfeedback mechanism 936L and a right vibrating feedback mechanism 736R.

These vibrating feedback mechanisms, alone or in combination, may beused to provide feedback to the user regarding the left or right sideactivation of energy to tissue. In the arm rest or at the mastercontrollers, a left side vibrating feedback mechanism may be activatedto indicate left side activation of energy supplied to the left handedtool (or that left side is ready to fire) and a right side vibratingfeedback mechanism may be activated to indicate right side activation ofenergy supplied to the right handed tool (or that right side is ready tofire). Alternatively, different vibrating patterns may be used todistinguish between left side activation and right side activation ofenergy, such as for the haptic feedback provided to a surgeon's foot.The surgeon could sense the difference in the vibrating patterns withhis foot. As discussed further herein, several different signal profilesof signals may be coupled to the vibrating feedback mechanisms togenerate vibrations for vibrating patterns.

Enhanced Audible User Interface for Electro Surgical Systems

In addition to visual feedback provided by the GUI and/or tactile/hapticfeedback provided by a vibrating device, auditory feedback may be usedto indicate the energy status of an electrosurgical tool to a user. Theaudible feedback, alone or in combination with visual and hapticfeedback, may be used to provide feedback to the user regarding the leftor right side activation of energy to tissue.

The surgeon's console 150, 150A may include one or more audiotransducers or speakers. In FIG. 3A, the surgeon's console 150, 150Aincludes a left speaker 315L and a right speaker 315R coupled to a soundgenerator 317. When the user/surgeon is properly positioned, his leftear is near the left speaker and his right ear is near the rightspeaker.

The sound generator may be a stereo sound generator to independentlycouple sounds to the left speaker 315L and the right speaker 315R. Inthis case, the left speaker generating a sound may be used to indicate aleft side activation of energy to tissue and the right speakergenerating a sound may be used to indicate a right side activation ofenergy to tissue. Alternatively, the sound generator may be a mono soundgenerator to couple the same sounds to the left speaker 315L and theright speaker 315R. In this case, different audible patterns may be usedto distinguish between left side activation and right side activation ofenergy to tissue, such as for the haptic feedback provided to asurgeon's foot. The surgeon could sense the difference in the audiblepatterns.

Enhanced Feedback Signaling

The type of feedback provided to a user at the surgeon's console 150,150A may have one or more different forms—visual feedback,haptic/tactile feedback, and/or audible feedback. The feedback may beseen by the user's eyes; felt by the user's feet, hands, and/or arms; orheard by the user's ears. The feedback itself may be of adistinguishable pattern and provide further information if differentsignal patterns/profiles are used.

Referring now to FIG. 12, different signal profiles of signals that maybe used to provide user feedback. In one case, the feedbackpattern/provide may be a constant vibration, a constant buzzing sound,or a constant visual flashing display in the GUI that a user can sense.In another case, the feedback may be periodic or metronome-like, such asa periodic vibration felt by the user, a periodic noise generated byleft and/or right speakers, or a periodic visual flash generated on thedisplay that a user may see.

To generate a constant feedback due to a vibration, a buzz or a flashingmay be generated by a sinusoidal curve 1201A, a square wave curve for1201B, or a triangular waveform 1201C. Each of these may have a fiftypercent (50%) duty cycle to generate the constant feedback. To generatea periodic feedback, such as a periodic vibration, periodic noise, orperiodic visual flash, a waveform coupled into the feedback transducermay have a periodic pulse such as the triangular waveform 1202A or theperiodic pulse waveform 1202B illustrated in FIG. 12. In this case, thewaveforms each have a duty cycle that is less than or greater than fiftypercent (50%). The feedback signals may be tuned or adjusted to suiteach user's individual preference. That is, the rate/frequency,amplitude, duty cycle, etc., can be tuned or adjusted to suit a userpreference in distinguishing between left and right activation, forexample.

The constant feedback and the periodic feedback are distinguishable andmay be used to indicate different feedback information to the user. Theperiodic feedback and the constant feedback may be used to distinguishbetween right activation and left activation of an electrosurgicalinstrument when equally coupled to one or more speakers 315L, 315R in amono audible system or one or more energy pedals 704R, 708R in the rightside of the pedal tray.

To further distinguish between left and right, the constant feedback(e.g., waveforms 1201A, 1201B, 1201C) may be coupled into a left mastercontroller 905L, a left pedal 704L, 708L, and/or a left speaker 315L toindicate left hand side energy activation for a left hand side tool;while the periodic feedback (e.g., waveforms 1202A, 1202B) may becoupled to the right speaker 315R, the right pedals 704R, 708R, and/orthe right master controller 905R to indicate right hand side activationof the right hand side tool. Alternatively, the periodic feedback may becoupled to the left side transducers to indicate left hand side energyactivation for a left hand side tool; while the constant feedback may becoupled to the right side feedback transducers to indicate right handside activation of the right hand side tool. In this manner, the user isfurther informed of the side of activation by the handedness of thefeedback.

For example, a persistent audio indication (e.g., a 1 Hz “metronome”click) may be coupled to the user's left or right ear by the respectivespeaker to indicate that the left or right hand side tool will bereceiving energy. Alternatively, a momentary audio indication (e.g.,clicks, buzzing) may be coupled to the user's left or right ear by therespective speaker when the energy is activated in the left or righttool. In another embodiment of the invention, a mono system, a metronomelike tone may be coupled into speakers of the surgeon's console toindicate left hand side tool activation energy while a momentary audioindication may be coupled into the surgeon's console to indicate righthand side tool activation energy.

A persistent haptic indication (e.g., a haptic “metronome” pulse) may becoupled to the left or right hand grip by respective vibrating devicesto indicate that the left or right hand side tool will be receivingenergy. Alternatively, a persistent haptic indication may be coupled toa foot pedal to differentiate from energy activation of the left handside from the right hand side. Alternatively, a persistent hapticindication may be coupled to the left portion or the right portion ofthe arm rest by respective vibrating devices to distinguish betweenenergy activation of the left hand side and energy activation of theright hand side. In another embodiment, a momentary haptic indication(e.g., click, buzz vibration) may be coupled to the master control (leftor right) by a respective vibrating device corresponding to theinstrument that receives the energy.

A persistent visual indication (e.g., a barber pole) may be overlaidonto a left or right active strip in the user interface by the graphicsgenerator to indicate that the left or right hand side tool will bereceiving energy. In another embodiment, a momentary indication (e.g., aperiodic flash of a left or right side active strip in the graphicaluser interface by the graphics generator may be used to indicate thatthe left or right hand side tool will be receiving energy.

The overall time period that the feedback signals are generated foreither side may vary on how it is desired that the feedback be conveyed.For example, the feedback signal may be constantly generated duringsurgery in the side that has the hot hand to energize an electrosurgicaltool. Alternatively, the feedback signal may be generated to providefeedback only while a user depresses an energy pedal to energize anelectrosurgical tool. In yet another alternate embodiment of theinvention, the feedback signal may be generated for an overallpredetermined time period after the user only touches the pedal. In thiscase, the foot pedals (such as the energy pedals) may include afeather-touch sensing device or an optical sensing device 727 as shownin FIG. 7D to detect when a user's foot is hovering over a particularpedal that may be integrated with their switches. In another embodimentof the invention, feedback may only be provided when a pedal is ready tobe activated or be pressed, such as to fire to couple energy intotissue. If no electrosurgical tool is present on the side selected, thepedal is not ready to be activated and thus feedback would not beprovided that would indicate it was ready. If an electrosurgical tool ispresent on the side selected but no generator is coupled to theelectrosurgical tool or the wrong energy is coupled to theelectrosurgical tool or being controlled by the foot pedal, the pedal isnot ready to be activated and thus feedback would not be provided thatwould indicate it was ready.

Enhanced Energy Activation Control

Methods to prevent or mitigate the effects of incorrect energyactivation are now described.

In the case where a user (surgeon, operator O) has forgotten or fails tocheck the left or right status of energy, it is possible that he or shewill fire the incorrect energy device by simply stepping on one of theenergy activation pedals. That is, electrosurgical energy may be appliedto the tool on the wrong side if the wrong side is selected forapplication of energy by the energy pedals. To avoid this, user feedbackmay be provided before the electrosurgical energy is applied to the toolso that the user may back off from firing the incorrect energy device.

Immediately upon pressing the energy pedal, the user may be given sidedfeedback (alternatively referred to as handedness feedback) indicatingwhich electrosurgical tool, the left side tool or the right side tool(respectively controlled by the left or right master grip), is going tofire. This sided feedback, as described previously, could be the visualfeedback, haptic feedback, audio feedback, or a combination thereof. Forexample, a buzzing sound and a flashing icon on the left hand or righthand side of display could indicate which handed energy tool, the energyinstrument controlled by the left hand or the right hand, is going tofire, before it does fire or as it actually does fire.

If, simultaneously, the energy is actually applied to tissue while thesided feedback is presented, some damage may possibly occur to patienttissue. If the sided feedback is rapidly presented to the user, it canallow a user to quickly cease application of energy and thus mitigateany possible damage. Moreover, a brief delay in the actual applicationof energy after the sided feedback is presented, may allow the user todetect the feedback and cease the mis-application of energy to tissuebefore it occurs. An application of energy to tissue by anelectrosurgical tool on the wrong side is an exemplary mis-applicationof energy.

Referring now to FIG. 13A, a flowchart diagram is illustrated forenhanced activation of an electrosurgical device in accordance with oneembodiment of the invention. The method begins at block 1300 then goesto block 1302.

At block 1302, a determination is made if the input device is activatedto apply electrosurgical energy to a tool. If the input device is notactivated, the process loops back around to block 1302 which is thenrepeated. If the input device is activated, the process goes to block1304. The input device in this case may be an energy pedal that ispressed upon by a user's foot to close a switch to eventually causeelectrosurgical energy to be applied to a tool with a selectedhandedness. The input device requests the application of electrosurgicalenergy in this case before it is to be applied.

At block 1304, the user is provided feedback of the activation of theinput device to request the application of electrosurgical energy. Theuser feedback may be in the form of that previously described. The userfeedback provides not only identification of activeness but thehandedness of the activation. Prior to applying energy to tissue throughthe electrosurgical device, a delay period may be provided to allow timefor the user to back off. The process goes to block 1306.

At block 1306, the system waits for a predetermined period of time(delay) before applying electrosurgical energy in order to allow theuser to receive the feedback and decide whether or not to cancel theactivation of an electrosurgical device. The process goes to block 1308.

At block 1308, a determination is made if the input device has beendeactivated. If so the process goes to 1313. If not, the process goes toblock 1310. In the case of an electrosurgical tool, a user may releasethe energy pedal by lifting up and taking his foot off the pedal toallow it to open the switch and deactivate the application ofelectrosurgical energy. The process then goes to block 1310.

At block 1310, assuming the input device was not deactivated, theelectrosurgical device is activated to apply energy to tissue of apatient. The process then goes to block 1312.

At block 1312, a determination is made if the input device has beendeactivated. If not, the process loops back around to block 1312 andcontinues to sense whether or not the input device has been deactivated.If it is detected that the input device has been deactivated, the usersfoot has been lifted off the pedal, the process goes to block 1313.

At block 1313, with the input device being deactivated, theelectrosurgical device is deactivated so that energy is not furtherdelivered to tissue. The process then loops back around to block 1302 tocontinue through the loop.

In this manner, each time that an input device is activated to causeelectrosurgical energy to be delivered to tissue, a check is madewhether or not a mistake has been made by the surgeon in activating theelectrosurgical device. However, a surgeon may only need be remindedonce and a check made once whether or not it is proper to activate anelectrosurgical device. In which case, the repetitive delay periodintroduced by block 1306 may be unnecessary after a first check has beenmade.

Referring now to FIG. 13B, a flowchart of a process for enhancedactivation of an electro surgical tool is illustrated in accordance withanother embodiment of the invention. In this process, there is aninitial check to determine if it is proper to activate anelectrosurgical device. Subsequently, if the electrosurgical tool isrepeatedly used within a predetermined period of time, there is no delayin the activation of the electrosurgical device. The process shown inFIG. 13B can be viewed as having a delayed activation loop with adelayed activation and a subsequent no delay activation loop. Theprocess begins with block 1300 and then goes to block 1302.

At block 1302, a determination is made if the input device is activatedto apply electro surgical energy to a tool. If the input device is notactivated, the process loops back around to block 1302 which is thenrepeated. If the input device is activated, the process goes to block1304.

At block 1304, user feedback is generated and provided to the user toindicate the start of the electrosurgical activation process. Theprocess then goes to block 1306.

At block 1306, a delay of a predetermined period of time is introducedbefore activation of the electrosurgical device. This delay allows sometime for the user to receive the feedback and make a decision todeactivate the input device to avoid tissue damage if miss-applied. Ifthe feedback agrees with the user selection, the user may not deactivatethe input device and waits the predetermined period of time beforeelectrosurgical energy is applied. After the delay, the process goes toblock 1308.

At block 1308, a determination is made if the input device has beendeactivated. If so, the process loops back around to block 1302. If not,the input device remains activated and the process goes to block 1310.

At block 1310, the electrosurgical device is activated such thatelectrosurgical energy is delivered to tissue at the end of the toolwith the selected handedness. The process then goes to block 1312.

At block 1312, a determination is made again if the input device isdeactivated. If not, the process loops around continuously to detectwhen the input device is deactivated. If it was detected that the inputdevice has been deactivated, the process continues and goes to block1313.

At block 1313, with the input device being deactivated, theelectrosurgical device is deactivated in response thereto. The processthen goes to block 1314.

At block 1314, a determination is made if the input device has beenre-activated. If so, the process goes to block 1318. If the input devicehas not been re-activated, the process goes to block 1316. The inputdevice may be one or more energy pedals that are pressed by a user'sfoot to request the application of electrosurgical energy to tissue.

At block 1316, a determination is made if a predetermined period of timehas elapsed prior to the reactivation of the input device. Thepredetermined period of time allows the system to determine if theelectrosurgical device is going to be repeatedly used or not. Anelectrosurgical tool is repeatedly used if the time period is shortprior to reactivation of the input device. If the input device isreactivated before the predetermined time period lapses, the processgoes to block 1318. If the predetermined time period has elapsed withoutthe input device being re-activated, the process goes back to block 1302to repeat the initial delayed activation loop and allow the user to backoff from input activation and the application of electrosurgical energyto tissue.

At block 1318, the user is provided feedback of the repeated activationof the electrosurgical device. The process then loops around back toblock 1310 where the electrosurgical device is activated once again. Theblocks 1310-1318 are repeated when the electrosurgical device is usedrepeatedly with short periods of time before reactivation.

Graphical User Interface Formation Methods

Referring now to FIG. 14, a flow chart is illustrated of a process forgenerating a graphical user interface and displaying the GUI with videoimages to provide visual feedback to a user. The process begins atprocess block 1400 and then goes to process block 1404.

At process block 1404, a first color bordered portion in a first side ofa user interface is generated in order to indicate an energy activationstatus of a first electrosurgical tool that is associated with the firstside. A second color border portion is generated in the second side ofthe user interface in order to indicate the energy activation status ofa second tool the type of tool associated with the second side. Forexample, as shown in FIG. 5A, the right side colored border, 502R, iscolored blue to indicate that the first tool is an electrosurgical toolthat may be activated by a surgeon. In contrast, the left color borderat 502R may be colored brown to indicate the lack of an electro surgicaltool (a surgical tool without energy capability) being associated withthe left hand and therefore may not be controlled to provide energy totissue. The process then goes to block 1406.

At process block 1406, an energy handedness assignment icon (an energyball icon 508) is generated in the first side or second side of the userinterface to indicate left or right handedness and an active or inactiveenergy control of the robotic surgical tool in the side selected be theuser. The process then goes to block 1408.

At block 1408, an iconic pedal map including pedal icons are generatednear a center of the user interface to indicate pedal control functionsof the surgical system. The iconic pedal map may include one or moreswap or control pedal icons. The iconic pedal map is a subset of mastercontrol icons that may be displayed by the graphical user interface. Theiconic pedal map may illustrate the control status of the pedals in therobotic surgical system. The pedal icons and their positions in thepedal map may be a scaled representation of the foot pedals that may becontrolled by the user's feet. The process then moves to block 1410.

At block 1410, an iconic master grip handle map including master gripicons is generated to indicate left and right active or inactive mastergrip control. As shown in FIG. 5A, the master grip handle map may be apart of the master control icons 510. The master grip icons may bescaled representations of the master grip input controls that may becontrolled by the user's hands. The process then goes to block 1412.

At block 1412, one or more text tabs may be generated to indicate thetype of surgical tools that are mounted and currently available to theuser for control during surgery. A first text tab in the first side ofthe user interface located at the first electrosurgical tool isgenerated to indicate its tool type. A second text tab in the secondside of the user interface associated with a second tool is generated toindicate its tool type. A third text tab in the first side or secondside of the user interface may be generated and associated with thethird tool to indicate the tool type to which a swap may occur. In FIG.5A, a left tool type text icon 505L, a right tool text icon 505R. and aswap tool text icon 509 are illustrated. The process then goes to block1414.

At block 1414, one or more tool icon numbers associated with control ofthe respective one or more surgical tools is generated. In FIG. 5A, aright tool number icon 504R, a left tool icon 504L, and a swap tool iconnumber 504S are generated in the user interface. The process then goesto block 1416.

At block 1416, a camera orientation icon is generated in the userinterface associated with the orientation of an endoscopic camera thatcaptures the images of the surgical worksite. This provides anindication of the orientation of the camera with respect to the imagesof the surgical site to provide a frame of reference. The process thengoes to process block 1418.

At process block 1418, the user interface is overlaid onto video imagesin the display device. The user interface is graphically generated andfused together with video images by the graphics generator 322 so thatit is overlaid on top of the video images. With the graphical userinterfaced fused together with the video images, pixel information ofimage frames may be sent to the display device for display.

Icons generated for the user interface may change in color (bright ordark color), position (left or right), and information content (activeicon/inactive icon). Moreover, the video images may change over time asmore frames of images are captured by the endoscopic camera. In whichcase, one or more of the process steps may repeated as need tocontinuously generate the graphical user interface overlaid onto thevideo images. If a surgery is completed, the camera is turned off andthe process may then go to block 1499 and end. While a number of theprocesses are shown and described as being performed serially, one ormore of them may be concurrently performed as if being performed inparallel.

Swapping Energy Handedness

Previously, swapping energy handedness or simply energy swapping wasdescribed with reference to FIG. 5A, as well as other figures. Themethod of energy swapping at the surgeon's console is now described withreference to the flow chart of FIG. 15.

Referring now to FIG. 15, a method of swapping energy handedness fromone surgical tool in one side to another surgical tool in an oppositeside is illustrated by a flowchart. The process begins at process block1500 then goes to process block 1502.

At process block 1502, a determination is made if an energy swap isrequested to swap the handedness of the energy control from left toright or right to left. An energy swap may be requested by the user bypressing on the energy swap pedal 706R. If an energy swap is notrequested, the process loops around back to block 1502 until the energyswap is requested by the user by way of the selection of the energy swappedal. If an energy swap has been requested (energy swap pedal has beenpressed) by the user so that energy control is to be passed from onehanded side to the other, the process goes to process block 1504.

At process block 1504, energy control handedness is swapped from a firstside to a second side in response to the selection of the energy swappedal. For example, the energy control handedness may be swapped fromthe right hand as shown by the user interface illustrated in FIG. 5A tothe left hand as shown by the user interface illustrated in FIG. 5C. Theprocess then goes to process block 1506.

At process block 1506, the energy-handedness assignment icon (energyball 508) is swapped from the first side to the second side of the userinterface to indicate the energy handedness swap to a user. For example,FIG. 5A shows the energy ball icon 508 in the right color border 502R.After the energy swap request from the user, the energy ball icon swapssides. As shown in FIG. 5C, for example, the energy ball icon 508″ is inthe left side border 502L after the swap. The process then goes to block1508.

At block 1508, determination is made if the second side has anelectrosurgical capable tool installed and if so, is there anappropriate electrosurgical generator under control of the second sideinput devices. If the second side has both an electrosurgical capabletool installed and an appropriate electrosurgical generator under itscontrol, then the process goes to process block 1510. If not, theprocess goes to block 1512.

At block 1510, assuming the second side has an electrosurgical capabletool installed and an appropriate electrosurgical generator iscontrolled by the second side input devices, then an active energyborder and an active energy-handedness assignment icon are displayed inthe second side of the user interface. An inactive energy border may bedisplayed in the first side of the user interface as the energy controlis now with the opposite side. The right side border 502R shown in FIG.5A illustrates an active energy border with an active energy ball icon508. After the swap, the right side border 502R′ shown in FIG. 5Cillustrates an inactive energy border.

At block 1512, assuming that the second side does not have anelectrosurgical capable tool installed or alternatively, there is not anappropriate electrosurgical generator controlled by the second sideinput devices, then an inactive energy border is displayed in the firstand second sides of the user interface. FIG. 5C illustrates an inactiveenergy border in both the left side border 502L and the right sideborder 502R′. Also, an inactive energy handedness assignment icon 508″may be generated and displayed in the user interface, such as shown inFIG. 5C.

Upon completion of the energy control swap from one hand to the otherand the display of the proper feedback to the user in the user interface1512, the process may go to block 1599 and end until the user selects tomake another energy control swap.

Adaptable Integrated Control of Remote Controlled Equipment

In a number of surgical procedures, surgeons want to be able to use onemore pieces of controlled equipment for robotic surgical tools inaddition to an electrosurgical generating unit (ESU) for a monopolar orbipolar electrosurgical tool. For example, they may want to use anultrasound tool or a laser capable tool that requires an ultrasoundgenerator or a laser generator as the controlled equipment. In order touse the extra controlled equipment, an external dedicated activationpedal was placed outside the surgeon console 150, 150A. Previously, thesurgeon had to move their foot off the pedals 318 to activate the extracontrolled equipment to provide a signal to the addition roboticsurgical tool. Movement of the foot away from the pedals 318 to controlthe external dedicated activation pedal is inconvenient and can resultin less efficiency during surgery as the surgeon looks away from themonitor down to his foot.

In order to provide increased flexibility for surgeons to use aplurality of ESUs as well as other remote controllable equipment withrobotic surgical instruments in a surgical procedure, an adaptableintegrated interface is provided between the between ESUs, activationpedals, and the robotic surgical instruments. The adaptable integratedinterface is programmable by software so that the remote controlledequipment is controlled by the pedals of the surgeon's console inresponse to the type robotic surgical instruments, controlled equipmentsupporting the respective instruments, and a user selection of the oneor more active tools. A surgeon can control multiple ESUs and othercontrolled equipment, such as suction pumps, irrigation pumps, andstapling devices, from the surgeon's console by an adaptable integratedinterface controller and integrated control system.

Referring now to FIG. 16, a block diagram of an integrated roboticsurgical control system 1600 is illustrated. At the heart of theintegrated control system is an integrated user interface controller1651. The integrated user interface controller 1651 may be located atthe surgeon's console 150 or the control cart 150B shown in FIG. 1B. Theintegrated user interface controller 1651 includes a plurality ofinput-output interfaces 1671A-1671N and a signal mapping device (alsoreferred to as mapping logic) 1672 coupled together. The integrated userinterface controller 1651 may further include a user input interface1681 coupled between the mapping device 1672 and the user interfaceinput devices 1601A-1601B to generate control signals in response toselection of the one or more foot pedal switches. The integrated userinterface controller 1651 may further include a user output interface1691 to couple to user interface output devices 1610A-1610B and aprocessor interface to couple to a processor (not shown, see FIG. 21) toinform a user of the status (e.g., active or hot-ready to fire- orinactive) of the robotic surgical tools by various user interfacetechniques described herein. The integrated user interface controller1651 may further include a tool interface 1692 coupled to the mappinglogic 1672 to communicate with one or more robotic surgical tools101A-101N or intelligent robotic surgical tools 101A′-101B′. The toolinterface may read tool information from the one or more roboticsurgical tools 101A-101N or intelligent robotic surgical tools101A′-101B′.

The integrated controller 1651 is coupled in communication with userinterface input devices 1601A, 1601B and user interface output devices1610A, 1610B to respectively receive input commands from the user andprovide feedback to the user. The integrated controller 1651 may also becoupled in communication with one or more electrosurgical generatingunits 102A′, 102B′ or other remote controllable equipment 1602. Theelectrosurgical units 102A′, 102B′ and the other remote controllableequipment 1602 are remotely controlled from the surgeon's console andmay be collectively referred to herein as remote controllable equipment1610. The integrated user interface controller 1651 may be in furthercommunication with one or more robotic electrosurgical tools 101A, 101Band/or other robotic surgical tools 101N coupled to the remotecontrollable equipment 1610.

The integrated user interface controller 1651 may receive informationabout the electrosurgical units 102A′, 102B′ and the other remotecontrollable equipment 1602 to which it is coupled in order to properlycontrol the equipment in its functional support of the robotic surgicaltools. To provide information about the remote controllable equipmentcoupled to the integrated controller 1651, one or more smart cables 1652may be used.

Smart cables 1652 are specially designed cables that furnish informationregarding the electrosurgical units 102A′, 102B′ and other remotecontrollable equipment 1602 to the integrated user interface controller1651. The smart cables 1652 couple between an activation port of the ESUor controlled equipment and an input/output interface 1671A-1671N of theintegrated controller 1651. The input/output interfaces 1671A-1671N ofthe integrated controller have software controlled switches that canadapt to the type of signaling needed to activate an ESU or othercontrolled equipment. The smart cables have unique identifiers thatidentify the ESU or other controlled equipment to the integratedcontroller so that the foot pedal switches of the surgeon's consoler areproperly mapped to perform the correct action, as if they were thestandard issue foot pedal provided with the ESU or other controlledequipment. The specially designed cables may also have furtherinformation, regarding the ESU or other remote controlled equipment thatit identifies, that can also be passed on to the integrated controller.

The smart cable 1652 may include a data cable 1661 and a control cable1662. The data cable 1661, which may not couple to the controlledequipment 1610, passes information regarding the remote controllableequipment 1610 to the integrated user interface controller 1651. Thecontrol cable 1662, coupled between the controlled equipment 1610 andthe integrated user interface controller 1651, allows the input/outputinterface 1671N of the integrated controller to emulate thefunctionality of the foot pedal switch the controlled equipment 1610 isexpecting. The emulation of the foot pedal switch in effect passescontrol signals from the integrated user interface controller 1651 tothe remote controllable equipment 1610 over the control cable 1662 whena foot pedal at the surgeon's console is activated.

Alternatively a bidirectional data cable 1654, such as an RS232 cable,may be coupled between the integrated user interface controller 1651 andthe remote controllable equipment 1610 so that information may bereadily passed bi-directionally between each. Instead of or in additionto the control cable 1662, control signals may also be passed over thebidirectional data cable 1654 from the integrated user interfacecontroller 1651 to the remote controllable equipment 1610 instead ofemulating the functionality of a foot pedal switch.

The integrated user interface controller 1651 may also be incommunication with the one or more robotic surgical tools 101A-101Nmounted to the robotic arms over electrical couplings 1621A-1621Nfacilitated by pins 424 of the tools coupled to terminals of theconnector 242 of the robotic arms. The integrated user interfacecontroller 1651 receives information about the one or more roboticsurgical tools 101A-101N to properly map and control the remotecontrollable equipment 1610 that supports the functions of the roboticsurgical tools. The one or more integrated circuits 426 in a tool storesinformation about the robotic surgical tool that is provided to theintegrated user interface controller 1651.

When a robotic surgical instrument that use a particular energy type ismounted to a robotic arm, it is identified to the integrated userinterface controller 1651 and matched with an appropriate piece ofremote controllable equipment, such as an ESU. A user interface input1601A, 1601B and a user interface output 1610A, 1610B may be provided toa surgeon at the surgeon's console 150, 150A such that they can selectwhich instruments they want to actively control with their left andright hands through the left and right master grips of the surgeon'sconsole. Some of the controllable features of the equipment and/or toolscan be mapped to one or more foot pedal switches.

The integrated user interface controller 1651 includes the plurality ofinput-output interfaces 1671-1671N and the signal mapping device 1672coupled together in order to map the one or more foot pedal switches ofthe one or more user interface inputs 1601A, 1601B to the remotecontrollable equipment 1610. In this manner, each piece of the remotecontrollable equipment 1610 may be selectively controlled by one or morefoot pedal switches. Some of the one or more foot pedal switches may beused to selectively control the kinematics of a robotic arm and tool,such as the endoscopic camera and the robotic arm to which it isattached. Another signal mapping device and additional input/outputinterfaces (not shown in FIG. 16) may be used to map the left and rightmaster grips to selectively control the kinematics of other robotic armsand the robotic surgical tools coupled thereto. The swap command canselectively alter the mapping of the left and right master grips toselectively control the kinematics of another robotic arm and roboticsurgical tool coupled thereto while another is idle.

The mapping can be automatically performed by the integrated controller1651 or it may be optionally selected by a surgeon using the userinterface inputs 1601A, 1601B. In either case, a surgeon can easilyoperate the remote controllable equipment 1610 and selectively controlthe supply of energy, fluids, gasses, etc. over the cables/hoses 1626 tothe robotic surgical tools 101A-101N and its application to tissue orthe surgical site by using the foot pedals and other provided by thesurgeon's console. Multiplexing the functionality of the pedals at thesurgeon's console vitiates the need for external independent pedalswitches to activate the remote controllable equipment. The userinterface output 1610A, 1610B informs the surgeon which robotic surgicalinstrument 101A-101N will be supplied with energy, fluids, gasses, etc.over the cables/hoses 1626 and which instrument will apply it within thesurgical site.

The integrated controller 1651 may be in communication with more thanone surgeon's console to receive the first user interface input 1601A ofa first surgeon's console and a second user interface input 1601B of asecond surgeon's console. Additionally, the integrated user interfacecontroller 1651 may provide the first user interface output 1610A forthe first surgeon's console and a second user interface output 1610B forthe second surgeon's console. The second surgeon's console may beoffsite located in a different room, building, city, and/or country fromthat of the first surgeon console.

The first and second user interface inputs 1601A, 1601B may each includethe one or more foot pedals 704L-704R, 706L-706R, 708L-708R, and themaster controllers 925L-925R, each having the switch 1010, as describedherein. The first and second surgeon's consoles may interchangeablyremotely control the remote controlled equipment and the roboticsurgical tools. For example, one or more foot pedals (of the first userinterface 1601A) of a first control console may be used to remotelycontrol a first piece of remote controlled equipment to supportfunctions of a first robotic surgical tool while one or more foot pedals1601B (of the second user interface 1601B) of a second control consolemay be used to remotely control a second piece of remote controlledequipment to support functions of a second robotic surgical tool.Alternatively, the first surgeon control console may control thekinematics of the robotic surgical tools while the second surgeoncontrol console may control the one or more pieces of the remotecontrolled equipment to support the functions of the robotic surgicaltools by controlling the supply of vacuum, gasses, liquids, and energyto the robotic surgical tools. A console swap signal may be generated bya switch (e.g., pedal switch 706L of the first control console) to mapthe appropriate foot pedals of either the first or second surgeonconsole to the remote controlled equipment.

The first and second user interface output 1610A, 1610B may each includea graphical user interface 501 that is displayed on one or moredisplayed devices 500 to provide visual feedback, one or more speakers315 to provide audible feedback, and/or one or more vibrating mechanismsat the master controller 925 or elsewhere on the surgeon's console toprovide haptic/tactile feedback. Other devices in the first and seconduser interface output 1610A, 1610B may be used to provided userfeedback.

Referring now to FIG. 17A, a simplified block diagram of the integratedrobotic surgical control system is illustrated to describe a firstmethod of operation regarding the control of electrosurgical tools andelectrosurgical units to provide energy to tissue. The integratedcontroller 1651 is coupled in communication with the robotic surgicaltools 101A, 101B; the electrosurgical units 102A-102C; and a lower rightfoot pedal 704R and an upper right foot pedal 708R. To provide aconsistent user interface input to the user, the lower right foot pedal704R switches a primary energy source on/off while the upper right pedal708R switches a secondary energy source on/off for a mono-polar ESU anda mono-polar tool, regardless of the handedness.

The integrated controller 1651 includes pedal control logic 1700A thatselectively controls how control signals from the foot pedals areselectively mapped into controlling the electrosurgical units. The pedalcontrol logic 1700A, along with the mapping logic 1672 and I/Ointerfaces 1617A-1617N, multiplexes control signals from the pedalswitches to control the one or more electrosurgical units to supplyenergy to the electrosurgical tools 101A-101B. The pedal control logic1700A may include the mapping logic 1672 that maps foot pedal controlsignals from the foot pedal switches to the appropriate controlledequipment 102A-102C. The I/O interfaces 1617A-1617N are represented by acore of switches 1718 in FIG. 17A.

The electrosurgical units 102A, 102C supply energy to theelectrosurgical tools 101A-101B over energy supply cables 1726A, 1726C.

Information from the smart cables 1622 or other cables is used toidentify the electrosurgical units to the pedal control logic 1700A. Therobotic surgical tools 101A-101B, when mounted to a robotic arm,communicate information regarding their tool type to the pedal controllogic 1700A over the connections 1621A-1621B. In this manner, the pedalcontrol logic knows the electrosurgical units 102A-102C and the roboticsurgical tools 101A-101B that are a part of the integrated roboticsurgical system 1600. The engagement of a tool to the robotic arm cantrigger a mapping of the foot pedals to remote controlled equipment. Inresponse to the information about the robotic electrosurgical tools101A-101B, the electrosurgical units 102A-102C, and one or more userselections, the pedal control logic 1700A multiplexes or maps thecontrol signals from the pedals to the appropriate electrosurgical units102A, 102C to provide energy to the appropriate robotic surgical tool asdesired by the user at the surgeon's control console. That is, themapping of the foot pedals to the remote controlled equipment is a hothand mapping and not just a functional mapping. The user selects whichof one or more robotic surgical tools mounted to the robotic arms is tobe active (selects the handedness) to receive a supply from the remotecontrolled equipment under control of one or more foot pedals in thesurgeon's console. If the remote controlled equipment is inappropriatefor the selected active robotic surgical tool, the foot pedals aredisabled (they will not fire) so that the remote controlled equipmentwill not supply the vacuum, gas, liquid, or energy to the roboticsurgical tool as a safety measure.

After the mapping is completed correctly, a foot pedal may be pressed toactivate a switch to signal to the remote controlled equipment to supplyvacuum, gasses, liquids, energy, mechanical torques, mechanical forces,data signals, control signals, etc. to one or more robotic surgicaltools.

In an alternate embodiment of the invention, the pedal control logicneed not be coupled to the electrosurgical units 102A-102C. Instead, acontrol signal path may be provided from the robotic surgical tools tothe electrosurgical units or other remote controllable equipment so thatthe tools control their own supply of electrical energy, gas, liquid,etc. from the ESUs or the other remote controllable equipment.

Referring now to FIG. 17B, a simplified block diagram another embodimentof the integrated robotic surgical control system is illustrated inwhich pedal control logic 1700B of the integrated user interfacecontroller 1651′ is coupled between the foot pedal switches 704L, 704R,708L, 708L and intelligent robotic surgical tools 101A′-101B′. In thiscase, the integrated user interface controller 1651′ need not be coupledto the remote controllable equipment 1610 (including the ESUs102A-102B). Thus, the communication interface between the integrateduser interface controller 1651′ and the intelligent robotic surgicaltools 101A′-101B′ may be simplified.

The intelligent robotic surgical tools 101A′-101B′ are similar to therobotic surgical tools 101A-101B but additionally include their owncontroller and optionally, an adaptable input/output interface 1771 togenerate control signals to control the remote controllable equipment1610 in response to a user's inputs from the user interface inputdevices 1601A-1601B. That is, the intelligent robotic surgical tools101A′-101B′ may act as a control signal relay to relay the commands ofthe user to the remote controllable equipment. The adaptable I/Ointerface 1771 may read equipment information from the one or morepieces of remote controlled equipment 102A-102B (alternatively from astorage device in one or more smart cables) and may adapt signal levelsto the control signal levels that the remote controlled equipmentdesires. That is, the adaptable I/O interface 1771 may emulate the footpedal switch ordinarily expected by the remote controlled equipment.

The plurality of foot pedals 704L-704R, 708L-708R are coupled to thepedal control logic 1700B to receive control signals from the user. Thepedal control logic 1700B communicates and is coupled to each of therobotic surgical tool 101A′-101B′ by means of a data/control connection1722A-1722B. A control signal path is provided by a switch control cable1724A-1724B coupled between the robotic surgical tools 101A′-101B′ andthe electrosurgical units 102A, 102B. The cables 1724A-1724B may be asmart cable including data lines to read data (e.g., equipmentinformation) and control lines or cables to control the remotecontrolled equipment (e.g., the electrosurgical generating units).Electrical energy generated by the electrosurgical units 102A, 102B whenactivated, is supplied from the ESU to the respective robotic surgicaltools 101A′-101B′ over one or more energy supply cables 1726A-1726B. Inthe case of a bipolar ESU and bipolar robotic electrosurgical tool, twoenergy supply cables are coupled between the ESU and the tool. Withdifferent types of remote controlled equipment, the cables 1726A-1726Bmay be referred to as supply cables to supply a vacuum, gasses,liquids/fluids, or energy (e.g., electrosurgical, laser, ultrasound) toa robotic surgical tool.

In an alternate embodiment of the invention, the robotic surgical tools101A′-101B′ may be standard the robotic surgical tools 101A-101B and thecables 1724A-1724B, 1726A-1726B, 1722A-1722B may couple to an adaptableI/O interface 1771′ in the robotic arms 153 to control the remotecontrolled equipment and the supply to the robotic surgical tools, asshown by the optical cables around the robotic surgical tool 101A′ inFIG. 17B.

The pedal control logic 1700B includes the mapping logic 1672 toappropriately map foot pedal control signals from the foot pedalswitches over the data/control connection 1722A-1722B to the appropriaterobotic surgical tool 101A′-101B′. Information regarding the roboticsurgical tools 101A′-101B′ is received by the pedal control logic 1700Bover the data/control connections 1722A-1722B. The data/control coupling1722A-1722B passes requests/control signals from the foot pedals to therespective robotic surgical tool 101A′-101B′. The robotic surgical tool101A′-101B′ passes the request/control signal to the respectiveelectrosurgical unit 102A-102B via a respective switch control cable1724A-1724B. In this manner, the robotic surgical tools themselvescontrol the supply of electrosurgical energy over the energy cables1726A-26B. Note that the switch control cable 1724A-1724B may be groupedtogether with the respective energy cable(s) 1726A, 1726B to provide asingle cable coupled between the remote controllable equipment 1610(ESU) and the robotic surgical tools 101A′-101B′.

Upon mounting the intelligent robotic surgical tools 101A′-101B′ to therobotic arms of the robotic surgical system, the system reads the storeddata in the tool to recognize the type of tool and properly communicatewith it and the corresponding remote controllable equipment.Additionally, the intelligent robotic surgical tools 101A′-101B′ mayhave a cable detection device or switch to detect when one of the cables1724A-1724B or 1726A-1726B is plugged into the tool. Upon detection,information regarding the remote controlled equipment, such as theelectrosurgical unit may be downloaded. Alternatively, the ESU may senda low energy signal that can be recognized by the instrument when theyare coupled together. The connectors of the tool may be polled bysoftware to detect the low energy signal indicating a connection hasbeen made. Information from the electrosurgical units may then bedownloaded and read by the intelligent robotic surgical tools.

The intelligent robotic surgical tools or instruments 101A′-101B′ maydirectly communicate with an electrosurgical generating unit ESU todirectly command it to energize and supply electrical energy to thesurgical tool. Alternatively, the intelligent robotic surgical tools orinstruments 101A′-101B′ may have a relay to control the activation ofenergy by the electro-surgical units or prevent the supply of energy toa tool in the case of a misconnection in the control cable. In eithercase, the electrosurgical generating units 102A-102B may have anintegrated control switch or signal translating unit 1728 coupled to theswitch control cable 1724A-1724B. The integrated control switch orsignal translating unit 1728 converts signals on the switch controlcable into a firing signal to control an electrosurgical unit 102A or102B or other controlled equipment. If no signal translation is needed,the unit 1728 may just be an input receiver to receive a control signalfrom the intelligent robotic surgical tool.

Referring now to FIG. 18A, a block diagram of the pedal control logic1700A shown in FIG. 17A is illustrated. To control the multiplexing ofsignals from the foot pedal switches to the remote controllableequipment 1610, the pedal control logic 1700A receives informationregarding the installed tool types from the one or more tools 101,information regarding the available remote controllable equipment fromthe equipment itself, and control signals from the user controllableswitches that affect how some foot pedal switches may control the remotecontrollable equipment. These user controllable switches may include afunction mode control switch associated with a foot pedal 706L, a swapswitch associated with the foot pedal 706R, and/or one or more masterswitches 1010 associated with the master grips of the mastercontrollers. In this case, the mapping of the foot pedal switches by thepedal control logic is context sensitive.

In particular, the energy swap switch of the swap pedal 706R (alsoreferred to as a control swap pedal) contextively controls the mappingor multiplexing of the control signals from the foot pedal switches tothe remote controllable equipment 1610. The energy swap switch of theswap pedal 706R determines which one of two electrosurgical toolsmounted to the robotic surgical arms is to have its remote controlledequipment controlled by the energy control foot pedals. The energy swapswitch of the right vertical pedal 706R alters the handedness of thefoot pedal control supplying energy to the electro-surgical tools,swapping between the left hand side and the right hand side. The energyswap switch of the right vertical pedal 706R generates a control swapsignal (also referred to as an active tool signal or an energy swapsignal for electro surgical tools and generators) to alter the mappingof the control signals from the foot pedal switches of pedals 704R, 708Rto the available electrosurgical generators for support of the availableenergy tools that is selected. For example, the control swap signal mayswap between active control of a first remote controlled equipment and asecond remote controlled equipment in the support (e.g., supply ofvacuum, gasses, liquids, energy, mechanical torques, mechanical forces,data signals, control signals, etc.) to a first robotic surgical tooland a second robotic surgical tool, respectively. Alternatively, it mayalter the control of the same piece of remote controlled equipment thatcan support two or more tools concurrently. For example, the controlswap signal may swap control of a first remote controlled equipment inthe support (e.g., supply of vacuum, gasses, liquids, energy, etc.) tothe first robotic surgical tool and the second robotic surgical tool.

A function/mode control switch in a left vertical pedal 706L (alsoreferred to as an arm or tool swap pedal) may also be used to controlthe multiplexing of the foot pedal control signals through the pedalcontrol logic to the electrosurgical tools 101. The function/modecontrol switch of the left vertical pedal 760L may generate a tool swapsignal to swap the kinematic control of one tool for another. Forexample, on a right hand side, one tool is active and under control ofthe master controller at the surgeon's console while another tool isinactive or idle. Electrical energy is typically not supplied to aninactive or idle tool that is not controlled by one of the mastercontrollers. With each tool swap between tools, the controlled featuresand the remote controlled equipment can change. Thus, with each toolswap signal to change control between swappable tools, the mapping ormultiplexing of the control signals from the foot pedal switches to theremote controllable equipment 1610 can change.

If a master switch 1010 is selected (such as to clutch one of the mastercontrollers), the pedal control logic may also disable one or morecontrol signals from the foot pedal switches in the foot pedals 704L,704R, 706L, 706R, 708L, 708R. In this case, one or more of theinput/output interfaces of the integrated controller may be disabled sothat an inadvertent press on a foot pedal does not supply energy to atool that is clutched or disengaged from a master controller.

The tool types of the one or more robotic surgical tools 101 alsoinfluences the mapping or multiplexing of the control signals from thefoot pedal switches. Each piece of remote controllable equipment 1610may require one or more control signals from one or more foot pedals forits features to be controlled properly. For example, consider anelectrosurgical tool that is mated to robotic surgical arm is a bipolarcollector surgical tool. In this case, only one foot pedal switch isneeded to activate the controlled equipment to supply electrosurgicalenergy to the bipolar tool. In another example, consider the installedtool type is a monopolar electro surgical tool. In this case, a pair offoot pedal switches may be used to control the remote controllableequipment to supply primary and secondary electrical energy to themonopolar electrosurgical tool. In either case, the pedal control logicsurveys all the remote controllable equipment 1610 coupled to it and allthe type of robotic surgical tools 101 mounted to the robotic surgicalarms in order to determine how to map control signals from the footpedals to control the remote controllable equipment. With smart cables,a surgeon can dynamically map the foot pedals to energy instrumentswithout explicit communication with the electrosurgical generating units(ESUs).

Referring now to FIG. 18B, a block diagram of the pedal control logic1700B shown in FIG. 17B is illustrated. Pedal control logic 1700Binterfaces to the foot pedal switches 704L-704R, and 708L-708R toreceive control signals that are multiplexed into the one or moreelectrosurgical tools 101 mounted to the robotic surgical arms, insteadof the remote controlled equipment 1610. Otherwise, the pedal controllogic 1700B is similar in that it may also receive information regardingthe installed tool types, the available remote controllable equipment1610, an energy/control swap signal from the pedal switch 706R, anarm/tool swap signal from the pedal switch 706L, and a function/modecontrol switch in order to multiplex the control signals towards theappropriate electrosurgical tool. The pedal control logic 1700B is alsocontext sensitive similar to the pedal control logic 1700A. In thismanner, a surgeon can dynamically map the foot pedals to energyinstruments.

The adaptable integrated controller allows a plurality ofelectrosurgical generating units (ESU) to be coupled to and controlledby a single surgeon console. The adaptable integrated controllervitiates a need for extension pedals outside of the normal workingfootspace of the surgeon's console to control additional remotecontrolled equipment of the system. The robotic surgical system can beset up with a proper mapping of the foot pedals to the remote controlledequipment and the robotic surgical tools prior to a surgical procedureto increase efficiency. The graphical user interface described hereinmay be displayed to the surgeon to communicate the state of the systemand how the energy pedals are mapped.

Software, hardware, or a combination thereof may be used to control theactive mapping of multiple remote controlled pieces of equipment (ESUs)to one or more instruments and one or more foot pedal switches.

Smart Cable

Referring now to FIG. 19, a perspective view of a smart cable 1652 isillustrated. The smart cable 1652 includes a first cable portion 1652Abetween a first connector 1910 and a data storage device 1902, and asecond cable portion 1652B between a second connector 1912 and the datastorage device 1902. The second connector 1912 of the smart cable 1652is configured to interface to a receptacle of one or more of the remotecontrollable equipment 1610. The first connector 1910 may be configuredto couple to the integrated controller 1651 or to the intelligentrobotic surgical tools 101A′-101B′. To avoid disconnections, the firstand second connectors may be lockable connectors, such as a bayonet or athreaded connector, that couple to bayonet or threaded receptacles,respectively.

The first cable portion 1652A of the smart cable has both data andswitch control lines. The data lines are routed between the data storagedevice 1902 and the first connector 1910 to read information from thedata storage device. As the input/output interfaces 1671A-1671N of thecontrollable equipment interface 2110 and adaptable input/outputinterface 1771 can adapt to the required signaling for each remotecontrollable equipment, there is no need for signal conversion in thesmart cable. The switch control lines may be routed between the firstconnector 1910 and the second connector 1912 bypassing the data storagedevice 1902. In this manner, the second cable portion 1652B of the smartcable 1652 may not have data lines but switch control lines alone.

The data storage device 1902 may be an integrated circuit chip with anon-volatile memory to store an identifying number or information of aparticular piece of remote controllable equipment 1610. For example, theidentification or information (e.g., make, model, remote controls,signal levels, etc.) may indicate the manufacture of the remotecontrollable equipment, the type of remote controllable equipment, andthe manner in which the remote controllable equipment may be controlledby the foot pedal switches of the foot pedals. The information in thedata storage device (also referred to as equipment information) providesa description of the remote controllable equipment (e.g., ESU) to whichthe smart cable is attached so that the user interface may adapt. Withthe addition of the tool information (e.g., instrument locationinformation, tool type information, etc.) from the tools mounted to therobotic arms, the foot pedals can be mapped to appropriately control theremote controllable equipment when a tool is active.

Alternatively, given remote controllable equipment 1610 with improvedcommunications capability, the smart cable may be replaced by abidirectional data cable, such as an RS232 cable for example. In thiscase, data lines may be coupled between the remote controllableequipment 1610 and the integrated controller 1651 so that data can bebi-directionally communicated. The integrated controller 1651 can thenpoll the remote controllable equipment for information that can be usedto set up and remotely control it to supply energy, gas, or liquids to arobotic surgical tool.

Adaptable Input/Output Interface

As described previously, the integrated controller 1651 has a pluralityof adaptable input-output (I/O) interfaces 1671A-1671N. Each of theintelligent robotic surgical tools 101A′-101B′ may include an adaptableinput output interface 1771 to couple to remote controllable equipment.The input-output interfaces 1671A-1671N, 1771 can adapt to the type ofsignal requirements needed by the remote controllable equipment 1610over the control lines of the smart cable. That is, signal levels formedby the closing and opening of the foot pedal switches can be adapted bythe adaptable I/O interfaces to signal levels to control the remotecontrolled equipment. Each of the I/O interface adaptors includes acontrol switch 2002. The control switch 2002 may be a transistorizedswitch or a mechanical relay type switch with a control input terminalto open and close the switch.

FIGS. 20A-20C illustrate how the switch 2002 may be adapted to differenttypes of control lines. Each input-output interface 1671A-1671N mayinclude one or more control switches 2002 in order to control one ormore features of the remote controllable equipment 1610. The connectionsof each switch 2002 may be made under software control to adapt to thetype of signaling needed to control the remote controlled equipment1610.

In FIG. 20A, a first remote controllable equipment is coupled to aninput-output interface 1671 by a first cable. A control line 2004A tothe remote controlled equipment is normally pulled up by a pull-upresistor R_(pu) to a pull up voltage Vc by the first remote controllableequipment. The switch 2002 has one pole coupled to the control line2004A and another pole coupled to ground. When the switch 2002 is closedby a control signal applied to its control terminal from a mapped footpedal, its poles are coupled together to pull down the control switchline 2004 to signal to the remote controllable equipment, as if a footpedal were directly attached to the control line 2004A.

In FIG. 20B, a second remote controlled equipment differing from thefirst is coupled to the input-output interface 1671 by a second cable.In this case, a control line 2004B is pulled to ground through apull-down resistor R_(pd). The switch 2002 has one pole coupled to thecontrol line 2004B and another pole coupled to a voltage source Vx. Inthis case when the switch 2002 is closed by a control signal applied toits control terminal from a mapped foot pedal, its poles are coupledtogether to pull up the control line 2004B to the voltage supplied bythe voltage source Vx.

In FIG. 20C, a third remote controlled equipment differing from thefirst and the second is coupled to the input-output interface 1671 by athird cable. A pair of control lines 2004C, 2004D are provided by thesmart cable to control the remote controllable equipment 1610. The pairof control lines are coupled to the poles of the switch 2002. Thecontrol lines 2004C and 2004D may be shorted together by a foot pedalswitch to signal the remote controlled equipment. In this case, thecontrol switch 2002 is coupled between the control lines 2004C, 2004D sothat when it is closed, it shorts them together. The control terminal ofthe switch 2002 receives a control signal from a respectively mappedpedal to open and close the switch.

Computer System with Integrated Controller

Referring now to FIG. 21, a block diagram of a computer system 151, 151Bwith the function of the integrated user interface controller 1651 isillustrated. The computer system 151B further includes one or moremicroprocessors 302, a scratch pad storage device 304A (memory), a disclike storage device 304B for permanent storage of software, a patientside cart (PSC) interface 2152, an audio generator 317, a graphicsgenerator (video card) 322, and a feedback generator 324 coupledtogether as shown. Each of the elements of the computer 151, 151B may becoupled to a central bus 2100 to communicate with one of themicroprocessors 302 or otherwise be directly coupled to a microprocessorvia a dedicated bus.

The integrated controller 1651 includes a remote controllable equipmentinterface 2120, a surgeon's console interface 2150, and a cross pointswitch 2170.

The remote controllable equipment interface 2120 interfaces the computerto the remote controllable equipment 1610, such as electrosurgicalgenerating units 102A′, 102B′, by means of smart cables, RS232 cables,or other communication channels.

The surgeon's console interface 2110 interfaces the computer to thesurgeon's console 150, 150A. The surgeon's console interface 2110couples to the user interface inputs 1601A, 1601B and the user interfaceoutputs 1610A, 1610B. The surgeon's console interface 2110 receivescontrol signals from some of the switches of the pedals and couplesthose into the cross point switch for mapping. The surgeon's consoleinterface 2110 receives control signals from some of the switches of thepedals and couples those onto the bus to be read by the microprocessorfor control of the cross-point switch. Other control signals, such asfrom the master controllers, are coupled onto the bus to be read andprocessed by the microprocessor and the software programs it executesfor controlling the system.

The cross point switch 2170, coupled between the surgeon's consoleinterface 2110 and the controllable equipment interface 2120, may beused to map the control signals from the foot pedals of the surgeon'sconsole to the remote controllable equipment. Otherwise, the cross pointswitch may be implemented in software or a combination of software andhardware. The pedal control logic 1700A, 1700B described previously maybe circuit logic, software stored on the disc storage device 304Bexecuted by the microprocessor, or a combination thereof to control themapping. With software, the microprocessor can control the cross pointswitch 2170 to perform the appropriate mapping of the foot pedals to theremote controlled equipment in response to information it receives aboutthe system.

The patient side cart (PSC) interface 2152 interfaces to the patientside cart 152 to control the kinematics of the slave robotic surgicalarms and the robotic surgical tools. The PSC interface 2152 receives thevideo images captured by the endoscopic camera and other sensor data orthe surgical arms and tools.

The feedback generator 324 is coupled to the audio generator 317 and thegraphics generator 322 to generate audible user feedback and/or visualuser feedback. The feedback generator 324 may generate feedback controlsignals for devices at the surgeon's consoler to provide tactile orvibratory user feedback. The feedback control signals may be coupledinto the devices at the surgeon's console through the surgeon's consoleinterface 1602.

Video signals from the graphics generator 322, fusing images of thesurgical site together with the graphics of the graphical userinterface, may be output to the surgeon's console through the surgeon'sconsole interface 2150 for display on a display device.

Audio signals generated by the audio generator 317, including audiosignals generated to provide audible feedback, may be coupled into thespeakers of the surgeon's console through the surgeon's consoleinterface 2150.

CONCLUSION

Some portions of the preceding detailed description have been presentedin terms of algorithms and symbolic representations of operations ondata bits within a computer memory or storage device. These algorithmicdescriptions and representations are the tools used by those skilled inthe data processing arts to most effectively convey the substance oftheir work to others skilled in the art. An algorithm is herein, andgenerally, conceived to be a self-consistent sequence of operationsleading to a desired result. The operations are those requiring physicalmanipulations of physical quantities. Usually, though not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared, and otherwisemanipulated. It has proven convenient at times, principally for reasonsof common usage, to refer to these signals as bits, values, elements,symbols, characters, terms, numbers, or the like.

It should be kept in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the above discussion, itis appreciated that throughout the description, discussions utilizingterms such as “processing” or “computing” or “calculating” or“determining” or “displaying” or the like, refer to the non-manual,automatic, or automated action and processes of a computer system, orsimilar electronic computing device, that manipulates and transformsdata represented as physical (electronic) quantities within the computersystem's registers and memories into other data similarly represented asphysical quantities within the computer system memories or registers orother such information storage, transmission or display devices.

The embodiments of the invention also relate to an apparatus or systemfor performing the operations described herein. This apparatus may bespecially constructed for the required purposes, or it may comprise ageneral-purpose computer selectively activated or reconfigured by acomputer program stored in the computer.

One or more elements in embodiments of the invention may be implementedin software to execute on a processor of a computer system. Whenimplemented in software, the elements of the embodiments of theinvention are essentially the code segments to perform the necessarytasks. The program or code segments can be stored in a processorreadable storage medium or device that may have been downloaded by wayof a computer data signal embodied in a carrier wave over a transmissionmedium or a communication link. The processor readable storage devicemay include any medium that can store information including an opticalmedium, semiconductor medium, and magnetic medium. Processor readablestorage device examples include an electronic circuit; a semiconductordevice, a semiconductor memory device, a read only memory (ROM), a flashmemory, an erasable programmable read only memory (EPROM); a floppydiskette, a CD-ROM, an optical disk, a hard disk, or other storagedevice, The code segments may be downloaded via computer networks suchas the Internet, Intranet, etc.

Note that the processes and displays presented may not inherently berelated to any particular computer or other apparatus. Variousgeneral-purpose systems may be used with programs in accordance with theteachings herein, or it may prove convenient to construct a morespecialized apparatus to perform the operations described. The requiredstructure for a variety of these systems will appear as elements in theclaims. In addition, the embodiments of the invention are not describedwith reference to any particular programming language. It will beappreciated that a variety of programming languages may be used toimplement the teachings of the invention as described herein.

While certain exemplary embodiments of the invention have been describedand shown in the accompanying drawings, it is to be understood that suchembodiments are merely illustrative of and not restrictive on the broadinvention, and that the embodiments of the invention not be limited tothe specific constructions and arrangements shown and described, sincevarious other modifications may occur to those ordinarily skilled in theart. For example, some embodiments of the invention were particularlydescribed with referenced to electrosurgical systems and tools but areapplicable to other types of remote controlled medical equipment andtheir respective surgical instruments.

What is claimed is:
 1. A user interface for a surgical system, the userinterface comprising: a display configured to output video images of aremote surgical site at which a plurality of instruments of the surgicalsystem are deployed; and a graphical user interface configured to beoutput on the display with the video images, the graphical userinterface comprising a visual indication indicating which instrument ofthe plurality of instruments is in a state of readiness for activationto deliver energy or actively delivering energy.
 2. The user interfaceof claim 1, wherein the graphical user interface is further configuredto output the visual indication at a portion of the display in proximityto where the instrument respectively appears in the display.
 3. The userinterface of claim 1, wherein the visual indication is positioned at aperiphery of the display.
 4. The user interface of claim 1, wherein thevisual indication comprises a change in appearance of the display. 5.The user interface of claim 4, wherein the visual indication comprises achange in appearance of the display located at a portion of the displayin proximity to a location of the display at which the instrument in thestate of readiness for activation or actively delivering energy appears.6. The user interface of claim 5, wherein the visual indicationcomprises a change in appearance of the display located at a relativeright portion or relative left portion of the display.
 7. The userinterface of claim 5, wherein the visual indication comprises a changein appearance of a portion of a border surrounding a periphery of thedisplay.
 8. The user interface of claim 4, wherein the change inappearance is chosen from at least one of a change of color, a change ofbrightness, a change in movement, and a change in pattern.
 9. The userinterface of claim 1, wherein the graphical user interface furthercomprises a number icon indicating a corresponding instrument of theplurality of instruments.
 10. The user interface of claim 1, wherein thegraphical user interface further comprises an additional visualindication indicating an orientation of an imaging device capturingimages of the remote surgical site that are output as the video imageson the display.
 11. The user interface of claim 1, wherein the graphicaluser interface further comprises an additional visual indicationindicating a type of each instrument of the plurality of instruments.12. The user interface of claim 11, wherein the additional visualindication is chosen from at least one of a color and alphanumericcharacters.
 13. The user interface of claim 11, wherein the additionalvisual indication comprises a symbol associated with energy.
 14. Theuser interface of claim 1, wherein the visual indication indicates whichof the plurality of instruments is in a state in which control over theinstrument by the surgical system can be swapped from another of theplurality of instruments.
 15. The user interface of claim 1, wherein thegraphical user interface further comprises additional visual indicationof one or more master control devices of the surgical system, the one ormore master control devices being operably coupled to operate one ormore functions of the plurality of instruments in response to input atthe one or more master control devices.
 16. The user interface of claim15, wherein the additional visual indication of the one or more mastercontrol devices comprises one or more icons chosen from pedals andgrips.
 17. The user interface of claim 15, wherein the additional visualindication of one or more master control devices comprises alphanumericcharacters indicating the one or more functions of the one or moreelectrosurgical instruments that the one or more master control devicesoperate.
 18. The user interface of claim 1, wherein the graphical userinterface outputs the visual indication in response to an input at amaster control device operably coupled to operate one or more functionsof one of the plurality of instruments in response to input at themaster control devices.
 19. The user interface of claim 1, wherein thedisplay is in communication with an image capturing device.